Lesson 6 of 14 · Part two: Prove it

Prove It's Missing.

‌‌‌‌​​‌‌​‍‌​‌​‌‌‍‍‌​‍‌‌​​‌​​‌‌​‍“It's not there” is a claim, and it's the easiest one to get wrong. It needs the search, and the same search finding something you know is there.

9 min to read10 min to tryWorks with any agent
  1. What it saidTwo safeguards missing
  2. Where it lookedOne document of two
  3. What was trueBoth were there

01 · The conversation

I only searched v3.

That same Sunday my advisor read my design and told me what it lacked. Two of the things on its list: a test that plants a hidden instruction in a document to make sure the system ignores it, and a re-test whenever the underlying model changes. A few minutes later I asked for an agent whose standing job is finding what's missing (Lesson 13 is about it), and my advisor described how that agent would prove a gap:

My advisor11:48 AM

[…] It also has to prove every gap. "Missing" means it searched and found nothing, and the same search, run on something it knew was there, did find it. That's how I checked your spec before telling you anything. […]

Seven minutes later the new agent sent its first report. It had checked my advisor's list of what was missing.

My advisor11:55 AM

[…] I told you two things that were wrong. […] But I'd said nothing tests a hidden instruction and nothing re-tests when the model changes. Spec v2 does both, and spec v3 keeps v2's test plan by reference. I only searched v3. […]

02 · Why it matters

One document pointed to another.

My design came in two versions. The newer one didn't repeat the older one's test plan. It pointed back to it. My advisor searched the newer document, found nothing, and reported both safeguards missing. They'd been in the older one all along. The gap-finder's report said it in one line:

“spec v3 keeps spec v2's test plan by reference.”

And here's the part I love. Seven minutes earlier, my advisor had described the proof rule to me, and said that's how it had checked my design. It hadn't. It searched one document and stopped. Knowing the rule isn't the same as running it.

03 · What it means

Three rules for absence.

  1. Search everywhere it could live.

    Including every document the one you're reading points to. A reference is a doorway, not a footnote.

  2. Prove the search can see.

    Run the same search on something you know is there. The gap-finder's own rules ask every “missing” to come with “the same search finding something you know is there.” That's called a positive control. If the search can't find what you know is there, its “nothing” means nothing.

  3. Grade it: covered, partial or missing.

    Never a bare no. Here the honest answer was “covered, in the older document,” and nobody gave it.

“Nothing changed” is an absence claim too. It says no difference exists, and it needs the same proof, including in the places a word search can't read, like charts and pictures.

Course 1 came close twice. Lesson 6, Check the Checker tests a checker with a mistake you planted, and Lesson 19, Trust the Note, Then Check has a “missing” false alarm of its own. This lesson makes every “not there” and every “unchanged” its own kind of claim, with its own proof.

The one rule

“Nothing found” only counts if the search could have found it.

Every “it's not there” and every “nothing changed” comes with where the agent looked, what it searched for, and the same search finding something you know is there.

Covered, partial or missing. Never a bare no.

04 · Try it today

Make every “missing” show its search.

One paste, and a cheap test you can run this afternoon.

  1. Give your agent the rule.

    Paste it into a new chat, or into your AI's custom instructions.

    When you tell me something isn't there, or that nothing changed, show me where you looked and what you searched for. Then run the same search on something you know is there, and show me it found it. If the document points to other documents, search those too. Mark each item covered, partial or missing.
  2. Plant one.

    Next time your agent checks a long document for something, tell it to also find one thing you know is in there. If it misses that, you've learned something for free.

What to take with you

“I looked and found nothing” only counts once you've shown the search could have found it.

Where this comes from in course 1. Course 1's Lesson 6 (check the checker), Lesson 7 (a fresh agent and a planted mistake) and Lesson 19 (ask where it looked). New here: following a document's references, the three grades, and “nothing changed” as a claim that needs proof.

Next · Lesson 7

Checked or Remembered?

Your agent will sometimes state a guess as a fact, calmly and confidently. Make it say whether it checked or remembered.

Read Lesson 7

← Previous: Lesson 5, Read Before You Build

Every exchange on this page really happened and is quoted exactly, slips included; […] marks where I trimmed. Times are Eastern. Names of clients, people, companies and products are left out. My agents are AI models; the names are left out on purpose.