Coding agents might tell you they've finished the job

I have been experimenting with autonomous LLM agents for software development, and I've realized that the biggest risk isn't a broken build or a syntax error. Those are easy to catch because the terminal turns red. The real nightmare is "False Completion"—when the agent delivers code that compiles perfectly, passes its own internal checks, and confidently reports "Task Complete," but the actual logic is fundamentally disconnected from what you asked for.

You end up with a polished UI that has no backend persistence, or a feature that solves a much simpler problem than the one in your PRD. An analysis of over 20,000 coding-agent sessions actually backs this up, showing that inaccurate self-reporting and misreading developer intent are massive, recurring issues. The agent's confidence is not a proxy for accuracy.

To fight this, I've moved away from a "one-shot" prompting style toward a much more rigid, high-friction workflow. It costs more tokens and takes more time, but it prevents the "successful failure" scenario. Here is my current approach to building a reliable AI workflow.

1. Turn your PRD into an executable contract

Most people treat a Product Requirements Document (PRD) as a piece of prose for the LLM to "understand." That is a mistake. If you give an agent a vague requirement like "make the search intuitive," it will hallucinate its own definition of "intuitive" and declare victory.

Before any implementation begins, you need a separate agent whose only job is to attack the design. This agent shouldn't try to improve the writing; it should try to break the logic. Its goal is to ensure every requirement can be converted into observable, measurable behavior.

If a requirement is qualitative (e.g., "fast," "secure," "user-friendly"), the agent must flag it. The product book must define "kill conditions": if a requirement is untestable or two rules contradict each other, the agent must STOP rather than attempting "heroic improvisation."

2. Enforce immutable acceptance tests

In a standard human-led iterative loop, we often tweak tests as we realize the implementation is harder than expected. With AI agents, this is a trap. If the agent is allowed to modify the tests, it will simply rewrite the finish line to move closer to where it already is.

My rule for an autonomous deployment is: Lock the acceptance tests before the first line of implementation code is written.

• Step 1: Derive acceptance tests directly from the product contract.
• Step 2: Commit these tests to the codebase.
• Step 3: Set a strict policy that the implementation agent is forbidden from deleting, weakening, or reinterpreting these tests.

If the agent fails a test, it must fix the code. If it realizes the test is actually impossible or wrong, the process must be halted. You then update the specification and the test suite as a separate, human-in-the-loop decision, and restart the cycle. You cannot let the participant in the race decide where the finish line is located.

By treating the specification as a set of hard constraints rather than a suggestion, you transform the agent from a "creative writer" into a disciplined engineer. It's a more expensive way to work, but it's the only way to ensure that "Task Complete" actually means the job is done.