Don't Grade Your Own Homework.
The agent that did the work shouldn't be the one that checks it. Hand it to a fresh agent that only gets the source material.
- Who built itMy agent
- Who checked itA fresh agent, sources only
- What it foundNothing made up. One wrong label.
“Don't do it yourself.”
In October a client asked us to prove that none of the numbers on their site were made up. Fair question. Their pages were full of figures, and my agent had built every page. It had even run its own careful check across the whole site already. I didn't want that check. Here's what I typed:
[…] perform a verification test to verify that we have not fabricated any data and that everything we have is wired to live API data […] so please spawn an agent. Don't do it yourself.
[…] I'm handing the verification to an independent agent that won't see my test or my conclusions, so it checks from scratch.
I've started a separate agent on […] verification test; it's still running. […] it can't change any files, deploy or send anything, and it reports back here.
Forty-seven minutes later:
[…] The verification came back clean. […] It flagged two wording errors, both now fixed […]
The agent that did the work already believes it.
Ask the agent that did the work to check it, and it reads its own work through its own assumptions. It knows what every number is supposed to mean, so that's what it sees. It's a student grading their own test with an answer key they wrote. I'm no better. I can read my own typo five times and never see it.
My agents know this about themselves, too. One Sunday night one of them sent me a numbered gallery of portraits and told me they looked like a real man. When one turned out to be a painting, it owned up to how it had checked:
“I hadn't looked at #11 before sending the gallery, and I told you the gallery read as a real man after reading only its top four pages. So my standard isn't proven either.”
Nobody was lying. It checked what it expected to find. A fresh agent doesn't expect anything. All it has is the work and the sources, so it has to look.
Clean. Except for one label.
The fresh agent got the finished pages and the original data, and nothing else. It traced every figure on the pages, thousands of points on the charts, every number in the articles and the data shown in a short video, each one back to its source. Then it did the thing I love most. It pointed a copy of the site at deliberately altered data and watched every figure change with it. That's Lesson 6 in action: it proved it could see before it told me everything was fine. Its report opened like this:
“I've finished the audit. Verdict: I found no fabricated performance data.”
And then the catch: “one table column has the wrong label.” The numbers in it were real. The heading said they counted one thing, and they counted another. Our own check had gone right past it, because we knew what that column was meant to show. The fresh agent only knew what the data said.
One more step mattered. My agent didn't take the finding and run. It checked the arithmetic itself, confirmed the fresh agent was right, and only then fixed the label. A second opinion can be wrong too. You check it before you act on it.
Notice what I didn't do: check thousands of numbers myself. My job was knowing the check had to come from somewhere else, and saying so. Writing the work is becoming the agent's job. Knowing who should check it is still yours.
Give it the sources, not the story.
A clean slate.
A new chat or a separate agent, with none of the conversation, none of the builder's notes and none of its conclusions. When my agent built a clip auditor for the avatar in Lesson 6, it kept that one blind on purpose: “To make sure it really learns your eye rather than just agreeing with labels it already knows, I'm keeping it blind”.
Only the work and the sources.
The finished thing, plus whatever it should match: the data, the documents, your original request. If the checker can't trace something back to a source, that's a finding too.
Hands off.
It reads and it reports. It doesn't fix, publish or send anything. Fixing is a separate step, after you've seen the list.
Check the findings.
Take the list back to the agent that did the work and have it confirm each item against the source before anything changes. That's how the wrong label got fixed, and how a wrong finding would have been caught.
Whoever made it doesn't get to sign it off.
When it matters (numbers a client will read, anything that goes out with your name on it), the agent that made the work is never the last one to look at it. Starting a fresh agent takes a minute. Finding a made-up number after it's gone out takes a lot longer.
Get a second opinion from a stranger.
- Finish something real with your agent.
A report, a summary of a long document, a spreadsheet, a page for your website. Anything with facts or numbers in it.
- Open a brand-new chat.
Or, if your agent can start helpers, ask it to start a separate one. Don't paste the old conversation, and don't explain how you got there.
- Hand over two things: the work and the sources.
Then paste this:
Here is a finished piece of work and the source material it was built from. Check every fact, number and name in the work against the sources. Don't fix anything. List what matches, what doesn't, and anything you can't trace to a source, and tell me how sure you are about each one. - Take its list back to the first agent.
Let it check each finding before anything changes:
An independent check found these issues. For each one, check it against the source yourself and tell me whether it's right before you change anything. - Notice what got caught.
Look at what the fresh agent found that the first one, and you, read right past. That's your answer to whether it was worth the minute.
The agent that did the work believes it. Hand the check to a fresh agent with only the work and the sources, keep it hands-off, and confirm what it finds before anything changes.
Never Guess
Your agent should infer what you meant, never what you decided. When your words could mean two things and it matters, it lays out the choices and asks. I learned that from the wrong Santa Claus. Twice.
Every exchange on this page really happened and is quoted exactly, slips included; […] marks where I trimmed. Times are Eastern. My agent here is Claude, made by Anthropic.