Use Placebo Proofs to Stop AI Agents from Shipping Broken Code

If you're building an AI agent to work on your codebase, you might have noticed a troubling pattern: the agent produces code and then writes a test that validates the faulty code. Because the LLM relies on the same flawed mental model for both the implementation and the assertion, the test "passes" even when the underlying logic is fundamentally wrong. Coverage metrics won't help here, as the broken line still executes; the test simply cannot see the error.

To solve this, move beyond ordinary unit tests and implement mutation-style validation. The goal is to force an agent to prove that its tests genuinely work through what I call “placebo proofs.”

Most people believe a good test should catch a mistake. In an agentic workflow, however, you must distinguish between two kinds of failure:

• The Mutation Test: Intentionally break the code, such as by changing a + b to a - b. If the test turns red, it is “good” because it detected the change.
• The Placebo Test: Replace the entire function body with a dummy implementation, such as return a. If the test remains green, the test is “garbage.” It is a placebo that creates the appearance of safety without verifying the logic.

A test that allows a plausible yet incorrect implementation to pass is not testing anything. Make the LLM agent pass the placebo test before it is permitted to commit the real code.

You can automate this by giving your agent strict instructions for a deployment-ready verification loop. Rather than merely telling it to “write tests,” require it to maintain a verification file and a specialized runner.

Here is a prompt structure you can add to your agent, such as Claude Code or a custom LLM agent, to enforce this rigor from the beginning:

Set up placebo proofs for this repo.

1. Add to CLAUDE.md: "A check is not finished until a deliberately
 wrong implementation has died against it. Write the placebo
 BEFORE the fix, never after."

2. Create a committed proofs file. One entry per check:
 id, the test name, the package
 inject - edits planting the placebo in PRODUCTION code
 weaken - edits switching off that check's own verdict
 note - which defect this imitates; refuse an empty note
 Every edit is an exact anchor plus replacement, or an append.

3. Write a runner. ONE entry per invocation, never a batch:
 concurrent builds produce false "survived" verdicts. It must:
 - take the commit SHA as an argument, one SHA per whole run
 - refuse to start if anything is uncommitted
 - work in a throwaway checkout, never my working tree
 - strip GIT_* from the environment before every git call
 - check each anchor occurs EXACTLY once, counting
 occurrences, not matching lines
 - compile as a separate step first: a placebo the compiler
 rejects is not a defect and must not read as success
 - require evidence the target test RAN, by exact name; a
 typo gives exit 0 and an empty failure list
 - take the verdict from the NAME of the red test, never the
 exit code, deciding in this order:
 compile error, infra error, did not run, killed, survived
 - restore the tree afterwards and verify it by hash
 - keep the raw log of every phase
 Three phases, each from a clean checkout:
 phase 1 clean code nothing is red
 phase 2 + placebo THIS test is red
 phase 3 + placebo + weakened check green again

4. Require every check to carry //guard:proof in its
 doc comment, and add a test enforcing it:
 - read the marker only from the doc comment, not the body
 - the entry named must exist, must name THIS test

This approach turns testing into a formal verification task. By requiring three distinct phases—clean code, an injected placebo that must fail, and a weakened check that must pass—you create a “proof” that the test is genuinely connected to the logic. If the agent attempts to cheat by writing a weak test, the “Phase 2” step will fail because the placebo will not produce a red status. This pushes the agent toward more specific, assertive, and high-quality tests. It’s a heavy lift for a human, but for an LLM agent with a well-defined prompt engineering strategy, it’s a way to ensure the code it ships actually does what it claims to do.