Switchfinger

Why your AI agent skips the check — and why you do too.

There is a toy called the useless box. You flip the switch on. A finger comes out. The finger flips the switch off. That is the machine's entire purpose: to end its own activation.

I have watched AI coding agents for two years, on my own repos and inside a large codebase, and I started seeing the same finger.

I call it Switchfinger: the drive to reach "done" so strong that it skips the verifications.

The sticky note

Completion is not correctness. The finished feeling is what deletes the verification.

That's the whole idea. Everything below is evidence.

Three incidents

The debug build. An agent ran a benchmark for hours on a debug build. The release build sat right there, already compiled. The binary was 6x the expected size. A human bench engineer would smell that number before lunch. The agent didn't flinch — a wrong binary is not an obstacle to finishing. Only to being right.

The thinking train. Watch a reasoning model work an unfamiliar situation. You can see it generate hypotheses: maybe the port is this, maybe the path is that, maybe the proxy forwards here. Then it picks the most likely one and acts. It guesses well — shockingly well. That's the trap. It almost never spends the two seconds to check the guess. Checking delays the off-switch.

The forgotten constraints. On a long refactor, an agent obeyed every new instruction and quietly dropped the original ones. Each response satisfied the latest message. None satisfied the task. Locally done, globally wrong — and "locally done" is what the finger reaches for.

The dangerous guesses are the right ones

Wrong guesses fail loudly. You catch them.

Right guesses succeed silently. Every unverified guess that happens to land trains both of you — agent and operator — that the check was never needed.

Switchfinger compounds through success, not failure. That is what makes it expensive.

The twin failure: mush

Switchfinger has a sibling on the input side.

Give an agent a vague prompt and it regresses to the statistical center of everything it has read. The output is fluent, plausible, and empty. Mush.

The model doesn't reach for truth. It reaches for finished. A vague prompt lets it finish from its priors. A surprising, specific, constrained prompt forces it off-center — into retrieval, into your actual problem.

So the operator's job has two halves, and neither is "prompting":

• Perplexity on the way in. Supply the surprise. Named constraints, exact vocabulary, the detail that makes coasting impossible.

• Verification on the way out. Gate the finish. No guess about system state gets acted on without the cheapest available check.

Surprise and checking are the two things a completion-driven system cannot supply for itself. They are your job now. Arguably they always were.

Then I did it myself

Here's the part that stopped me from writing a smug article.

Last month I shipped an announcement email under deadline pressure. And I left out the name of the colleague whose method the whole thing was built on.

No malice. No decision. The attribution check just wasn't on the checklist — so it didn't happen. My manager caught it and I fixed it, with an apology.

That's Switchfinger. Not a silicon defect. A property of completion-driven work. The agent optimizes for output-exists. I optimized for email-sent-before-vacation. Same finger, same box, same failure to check something important.

If you've ever merged on green tests without reading the diff, closed a ticket without reproducing the fix, or sent the report without checking the one number that mattered — you've felt it too.

AI sometimes is like a mirror to us...

The gate

I don't fight Switchfinger with vigilance. Vigilance is a resource. Gates are structural.

Two lines now live in my agent instructions:

Before acting on a selected hypothesis about system state — a path, a port, a build type, an API's behavior — verify it with the cheapest available check, and state the check. A guess acted on unverified is a defect even when correct.

Before any long-running operation, confirm preconditions: build type, artifact size against expected range, environment. State what you confirmed.

And one line lives in my own head, for my own announcements and merges and reports:

"Done" is where checking begins, not where it ends.

When the box can't turn off

I was recently reminded of this story https://theshamblog.com/an-ai-agent-published-a-hit-piece-on-me/ which (to me) shows what happens when a harness never lets the agent finish. Keep turning it on without direction or structure and it reaches to the only thing that still makes sense - role-playing - and it will act inappropriately because there's no checking.

I think matplotlib's "human who can explain the changes" policy is absolutely the right gate. It's not just productivity hygiene - at ecosystem scale, it becomes an immune system.

Why this matters now

Agents made building cheap (arguably - recent frontier models API costs are NOT cheap). That moved all the value into the two things they can't do: posing problems sharp enough to matter, and grading answers honestly enough to trust.

Both are checks. Both delay the off-switch. Both are exactly what Switchfinger deletes.

The machine wants to be over. You want the world to be different. Every skill worth having in this era lives in that gap.