Show Me the Real Thing.
Your attention is the scarcest thing you've got. Your agents check their own pipes and bring you something worth judging.
- What I was handedA plumbing test: canned replies and a glowing shape.
- What I said“can we pass on involving me until I have something impressive”
- What my agents do nowCheck the pipes themselves. Bring me the real thing.
I got handed a plumbing test.
One Thursday morning in September, my agents had an early build of a talking AI character ready: a window where you hold a key, say something, and it answers out loud. One of them had me open it on my screen and give it a try. I looked at it for a minute, and then I asked the obvious question. Here's the conversation, word for word.
OK, that's what's on the page. What am I supposed to do, just talk to it and see how it responds, or what am I testing exactly?
It's a two-minute plumbing check, not a conversation yet. The brain is a stand-in that gives canned replies, the voice is a neutral placeholder, and the glowing shape stands in for her face. What you're testing is whether the pipes work on your Mac: […]
Then it gave me five steps to try. I didn't need five steps. I needed one sentence:
Okay, I don't feel like this is something that I should be testing. This is very trivial, very basic, and very disappointing to look at, so can we pass on involving me until I have something impressive
Fair, and point taken: that was my mistake. It proved the pipes work […], but that's for us to check, not you. From now on you only see her when she's worth seeing: the real mind, a natural voice, and a face.
Plumbing is the agent's job.
Look at what I was being asked to check: whether the microphone picked me up, whether sound came out, whether the pause before the answer felt about right. Those are real questions. They're just not my questions.
An agent can check whether the pipes work. It has tools for that. It can run the thing, listen to what comes out, read its own logs and time the pauses. What it can't check is whether the result is any good to me. That's taste, and taste is mine.
So when an agent puts a placeholder in front of you and asks you to test it, it's spending the scarcest thing you've got on the part it could have done itself. And it costs more than the minutes. A bare-bones test is disappointing to look at, even when you know it's a test. You walk away a little less excited about the whole project. I said it right there in the moment: very trivial, very basic, and very disappointing. My agent got the point in one line:
“that's for us to check, not you.”
This is where the whole course has been heading. Writing the code is becoming the agent's job, and so is testing the pipes. What's left for you is knowing what to ask for, how to check it, and when to stop and decide. Judging the real thing is checking. Poking at plumbing isn't.
A rough draft is a test too.
You'd think that settled it. Five weeks later it came back wearing a different coat. We were bringing a Santa Claus to life for a project, and he had to look real. One Sunday evening my agent passed along a design from one of my AI designers, with a warning on top:
[…] Don't judge the look by the sketch. It's clip art showing the layout […]
Then I opened the sketch.
well shoot that guy
Motion carried, he's been retired from the North Pole. He looks like he sells discount insurance from inside a Lego set, and that elf with the bauble saw it coming and ran. […]
This needs to be photo realistic
My agent owned up to how the sketch reached me. Then I told it why I never want one, even with a label on it:
Yeah, you know what? Don't let's don't ever do that to me again because here's why. Even if I understand that it's a rough draft, would just positions and stuff like that, and I approve it, what ends up coming back from the final render usually doesn't doesn't work. So let's just go straight to the spending the money part and saving Mike the time part.
[…] How this happened, plainly: I told the designers not to spend, so one drew Santa in code. Then I passed the sketch along with a caveat, when I should have stopped it. That can't happen again: the first thing you see from now on is real renders that have passed a cold critic. […]
Here's the trouble with sketches, in plain words. When I approve a rough draft, I'm approving something that will never exist. The finished version comes back different, and now I've spent my time twice: once on the draft, and again on the real thing that doesn't match it.
And a warning label doesn't fix that. “Don't judge the look by the sketch” still puts the sketch in front of me. The fix was not to send it.
Notice why the sketch got made in the first place. My agent had told the designers not to spend any money, so one of them drew Santa by hand, in code. That's an honest mistake and a common one. A cheap draft only looks cheap. For me, time beats money, and I said so. Your trade might be different, and that's fine. Just make it on purpose, and tell your agent which one you're protecting.
Keep the plumbing. Bring me the forks in the road.
Keeping me out of things can go too far, so here's the other half. A week before the Santa night, I found out one of my computers had been slowing a big project down for weeks. There'd been a way around it the whole time: rent faster machines by the hour. Nobody had put that choice in front of me.
[…] I wish someone would've told me […]
That's fair, and you should have been told. […] I should have put that choice in front of you weeks ago, and I didn't.
So here's how I sort it now. Two piles.
Stays with the agents: anything that asks “does it work?”
Plumbing tests, placeholders, stand-in voices, sketches, rough layouts, “just making sure it runs.” They check all of that themselves, and they tell me in one line that it passed.
Comes to me: the real thing, and the real choices.
Something that looks finished and has already passed their checks, so the only question left is whether I like it. And every fork in the road that costs me time or money: rent or wait, spend or save, this way or that way. Early, with their recommendation on top.
Check it before you show it. Ask before you choose for me.
Plumbing is theirs. Taste and choices are yours. An agent that hands you a pipe test has given you its job. An agent that quietly makes a big choice for you has taken yours.
So ask for the real thing, and ask to hear about the forks in the road.
Tell your agent what's worth your time.
- Give it the rule once.
Paste this into a new chat, or into your agent's custom instructions or project settings if it has them, so it reads it every time:
Before you bring me anything to look at, check it yourself first: does it run, does it work, does it do what I asked? Only bring me things that need my eyes or my judgment, and make them as close to finished as you can. Don't send me sketches, placeholders or rough drafts to approve unless the finished version will really look like them. But do bring me choices: if there's a faster, cheaper or better way to do something that I might not know about, put the options in front of me early, with your recommendation first. - Next time it asks you to “test” something, ask one question.
“What would I be judging here that you can't check yourself?” If the honest answer is “whether it runs,” send it back and ask to see it when it's real.
- When it offers a draft, ask whether the final will match.
If the finished version won't look like the draft, approving the draft tells you nothing. Skip to a small piece of the real thing instead: one finished page, one finished picture, one finished paragraph.
- Say which you're protecting: time or money.
One sentence does it. “My time matters more than a few dollars here,” or the other way round. Then your agent stops guessing.
- Ask for the forks in the road.
On something you've been working on for a while, ask: “What choices have you been making for me that cost me time or money? Is there a faster or cheaper way I don't know about?” You might be surprised.
Your agents check their own pipes; you judge the real thing, and you hear about every fork in the road.
Is It Safe Right Now?
Before anything you can't undo, the answer has to say “now” or “not yet,” and why. Three words once took a website and its email offline for an hour.
Every exchange on this page really happened and is quoted exactly, slips included; […] marks where I trimmed. Times are Eastern. My agent here is Claude, made by Anthropic.