How to stop your AI agent from shipping broken code with placebo

If you are building an AI agent to handle your codebase, you have probably noticed a terrifying pattern: the agent writes code, then writes a test that perfectly validates that broken code. Because the LLM uses the same flawed mental model for both the implementation and the assertion, the test "passes" even when the logic is fundamentally wrong. Coverage metrics won't save you here because the broken line is still being executed; the test is just blind to the error.

To fix this, you need to move beyond simple unit tests and implement a mutation-style validation. I've been thinking about how to force an agent to prove its tests actually work by using what I call "placebo proofs."

The difference between a mutation and a placebo

Most people think a good test is one that catches a mistake. But in an agentic workflow, you need to distinguish between two types of failures:

• The Mutation Test: You intentionally break the code (e.g., changing a + b to a - b). If the test turns red, the test is "good" because it caught the change.
• The Placebo Test: You replace the entire function body with a dummy implementation (e.g., return a). If the test stays green, the test is "garbage." It’s a placebo—it provides the illusion of safety without actually verifying the logic.

A test that lets a plausible but wrong implementation slide through isn't testing anything. You need to force your LLM agent to pass the placebo test before it's allowed to commit the real code.

Setting up a robust AI workflow for proofing

You can actually automate this by giving your agent a strict set of instructions for a deployment-ready verification loop. Instead of just saying "write tests," you should instruct the agent to maintain a verification file and a specialized runner.

Here is a prompt structure you can drop into your agent (like Claude Code or a custom LLM agent) to enforce this rigor from scratch:

Set up placebo proofs for this repo.

1. Add to CLAUDE.md: "A check is not finished until a deliberately
 wrong implementation has died against it. Write the placebo
 BEFORE the fix, never after."

2. Create a committed proofs file. One entry per check:
 id, the test name, the package
 inject - edits planting the placebo in PRODUCTION code
 weaken - edits switching off that check's own verdict
 note - which defect this imitates; refuse an empty note
 Every edit is an exact anchor plus replacement, or an append.

3. Write a runner. ONE entry per invocation, never a batch:
 concurrent builds produce false "survived" verdicts. It must:
 - take the commit SHA as an argument, one SHA per whole run
 - refuse to start if anything is uncommitted
 - work in a throwaway checkout, never my working tree
 - strip GIT_* from the environment before every git call
 - check each anchor occurs EXACTLY once, counting
 occurrences, not matching lines
 - compile as a separate step first: a placebo the compiler
 rejects is not a defect and must not read as success
 - require evidence the target test RAN, by exact name; a
 typo gives exit 0 and an empty failure list
 - take the verdict from the NAME of the red test, never the
 exit code, deciding in this order:
 compile error, infra error, did not run, killed, survived
 - restore the tree afterwards and verify it by hash
 - keep the raw log of every phase
 Three phases, each from a clean checkout:
 phase 1 clean code nothing is red
 phase 2 + placebo THIS test is red
 phase 3 + placebo + weakened check green again

4. Require every check to carry //guard:proof in its
 doc comment, and add a test enforcing it:
 - read the marker only from the doc comment, not the body
 - the entry named must exist, must name THIS test

Why this works for LLM agents

This approach turns testing into a formal verification task. By requiring the agent to go through three distinct phases—clean code, injected placebo (which must fail), and weakened check (which must pass)—you are essentially creating a "proof" that the test is actually wired to the logic.

If the agent tries to cheat by writing a weak test, the "Phase 2" step will fail because the placebo won't trigger a red status. This forces the agent to write more specific, assertive, and high-quality tests. It’s a heavy lift for a human, but for an LLM agent with a well-defined prompt engineering strategy, it’s a way to ensure the code it ships actually does what it claims to do.