Check the Checker.
An automatic check only catches what it was built to look at. Before you trust one, test it on something you already know is wrong.
- What the check measuredHow much the mouth moved
- What I saw in one look“She's talking”
- What the new check asksAre the lips making words?
The checks said fine. My eyes said no.
We were building a talking avatar: a digital presenter you can have a live conversation with. My agents had built a whole set of automatic checks for it. Then one Sunday evening I sat down and actually talked to it for a minute.
Okay, I just did a brief test. Are we and by we, I mean you, aware that she seems to be playing video clips and this is not one continuous piece and every time she answers, it appears to start a new video clip. She literally jumps to a new position every single time. Did we miss this with our auditors?
↑ “She” is the avatar. Every time it started an answer, its face jumped.
Did we know? Partly. We knew her face is drawn in pieces of about a second each, and that the joins show; […] But the jump you're describing, a new position every time she starts an answer, is something our tests never measured. Our face rounds score long renders made off-line, not the live moment when she goes from listening to speaking on her page. So yes, the auditors missed it, and so did I.
[…] I'll also add it to the tests so it can't slip past again.
The next day we were making short clips of the avatar just listening: quiet, eyes on the camera. The automatic mouth check passed most of them. My agent asked me to pick the good ones.
None of them look good. She's talking, and all of them. Create an auditor to do the job you're asking me to do right now, please. It seems very important we get this right
Agreed: if she's listening, she isn't talking. That's now the auditor's first rule, and any speech-like lip movement, however small, fails a clip. Our mouth check missed the talking you saw because it measured how much her mouth moved overall, not whether her lips were forming speech. All 30 clips are rejected, and there are no more renders until the auditor works.
[…] How it proves itself: it has to judge clips it wasn't built from correctly before it judges anything new. If it can't tell talking from listening reliably, it'll say so instead of guessing.
A check is a ruler, not an eye.
Look at what those two misses have in common. Nothing was broken. Every check did exactly what it was built to do. The first set scored videos made ahead of time, and the jump only happened live. The mouth check measured how much the mouth moved, and a mouth can make words while barely moving at all.
That's the trap. A passing check doesn't mean “this is good.” It means “the one thing I measure looks fine.” Everything it doesn't measure sails right through, and it says PASS in the same cheerful green either way.
Here's the part I love. When my agent built the new auditor, it came back and told me where its eyes ended. It could clearly catch talking in some clips. In seventeen others it was going on tiny head movements, and it wasn't sure those meant talking. In its words: “So measurement alone can't yet see what you see in those 17.” That's a checker worth having. It knows what it can't see, and it says so.
This is the second job that stays human. Agents write the work, and agents can run the checks. Knowing what a check should look at starts with your eyes.
Don't just fix the mistake. Fix whatever missed it.
I'd learned this back in September. I spotted a three-second glitch in a short video we'd made with AI. Our AI editor, an agent whose whole job is to watch every cut before I do, had let it through. The obvious move was to have my agent fix the video. I said no.
[…] I don't want you to fix this. I want [our AI editor] to have noticed this so we need to fix [our AI editor] and then fix the video.
Understood: the gate failed, so the gate gets fixed first, then it sends the film back, then the film gets fixed through the loop.
If you only fix the video, you've fixed one video. The checker still has the same blind spot, and it'll wave the next one through too. Fix the checker, make it catch the problem on its own, then fix the work. Now that mistake gets caught every time, not just this once.
Test it on something you already know.
So how do you find a checker's blind spot before it bites you? You give it cases where you already know the answer.
One Sunday night in November, an AI critic was sorting a batch of AI-made Santa Claus portraits for me. It praised one for its realistic skin. I took one look: it wasn't a photograph at all. My agent worked out why. The critic “never saw a real photograph, so it had no anchor and took more detail for more real.” So I told it what I actually look for:
“We need to teach our critics how I see. which is really not that hard. […] Does someone's eyes matched their smile. The eye smile as much as the lips do. And that carries sincerity.”
My agent wrote that into a brief for every image critic we run. Then it said the honest thing: “No critic has been tested against it yet.” Writing a standard down isn't the same as proving a checker meets it. So the brief now opens every critic's job with a test:
“Real photographs and known failures go in the pack blind as controls. A critic that can't sort them doesn't count.”
The avatar auditor got the same treatment. Before it could judge a single new clip, it had to call the thirty I'd rejected “talking.” Your eye is the answer key. A checker earns your trust by agreeing with you on the cases you already know, before it goes to work on the ones you don't.
A check that has never failed hasn't been tested.
If a checker has only ever said PASS, you can't tell whether everything is fine or it simply can't see. Make it fail once, on purpose, on something you planted. After that, its PASS means something.
Plant a mistake. See if it gets caught.
- Pick one check you lean on.
A review prompt you reuse, a spell-checker, an agent you ask to “double-check this,” a test in your code. Anything that tells you it's fine.
- Ask what it can't see.
Ask your agent to be specific about the check's limits:
Before I trust this check, tell me exactly what it looks at, and what kinds of mistakes it would miss. Be specific. - Plant a mistake you already know about.
Take something that's right and break it on purpose: a wrong total, a misspelled name, a date that doesn't exist, a link to nowhere. Make one mistake obvious and one subtle. Don't tell the checker.
- Run the check on the good version and the broken one.
It should pass one and flag the other. If it misses the subtle mistake, you've found where its eyes end, at your desk instead of in front of a customer.
- When something slips past, fix the checker first.
Next time a real mistake gets through, say this before anything else:
This got past the check. Don't fix the work yet. First change the check so it catches this on its own, and show me it does. Then fix the work.
A pass only means the thing it measures looks fine. Test your checker on something you already know is wrong, and when a mistake slips past, fix the checker before the work.
Don't Grade Your Own Homework
The agent that did the work shouldn't be the one that checks it. Hand it to a fresh agent that only gets the source material.
Every exchange on this page really happened and is quoted exactly, slips included; […] marks where I trimmed, and [our AI editor] stands in for an internal name. Times are Eastern. My agent here is Claude, made by Anthropic.