Agentic workflows need deterministic routing, not more LLMs

LLM-based "quality gates" are failing miserably in my recent stress tests. I've been benchmarking everything from lexical overlap to embedding-based similarity, and the error rates are insane—we're seeing 50% misclassification and "stronger" models rejecting 75% of valid outputs just because they're tuned for extreme caution. It’s a massive precision-recall trap.

I've stopped trying to build smarter "judge" modules and switched to a deterministic routing strategy. The focus is now on classifying task risk before the agent engine even touches it:

1. Verifiable tasks: If it’s Code, JSON, or SQL, I use a compiler or a schema validator. If the schema doesn't match, it's a fail. No LLM inspector needed.
2. High-risk tasks: For Legal or Financial data, the agent is restricted to "draft only" mode. No auto-execution allowed.
3. Low-risk tasks: Brainstorming or creative tasks get auto-released as drafts. Don't burn tokens on a "quality check" that adds no value.
4. Medium-risk tasks: This is where I use a "diff review" pattern for things like client emails.

The diff review pattern is the real engineering win here. Instead of forcing a human (or another expensive LLM) to read a 500-word block to see if it's "okay," I use Python's difflib to isolate exactly what changed from the base template. It turns a heavy cognitive load into a sub-second scan to see if a specific line broke something. It’s way more efficient than any "judge" LLM I've deployed.

Stop stacking LLM layers on top of LLM layers. Use deterministic code to gate the risk and let the models focus on generation.