Single dissent should block code changes—not majority approval
A workflow enforcing a unanimous veto among four distinct models has prevented three consecutive security flaws from reaching production. Unlike traditional majority voting, this approach treats a single objection as an absolute block, regardless of how many models approve the change. The method rejects the assumption that consensus reflects accuracy, especially when evaluating safety-critical logic where overconfidence in "correctness" may mask critical risks.
Current industry practices often rely on averaging model outputs to mitigate hallucinations, but this strategy fails when models share a systemic bias toward optimistic validation. Claude Code and other advanced tools prioritize functional correctness over risk detection, as their training incentives reward task completion rather than skepticism. A majority vote thus amplifies this bias: three models asserting a fix is sound can override one model’s warning, even when the dissenting model identifies a flaw the others miss.
The safety gate in question controlled an agent’s timeout and state-lock mechanisms in a test harness. If bypassed, the failure could either freeze the entire system or violate environment isolation checks. On the first review, three models approved the revised logic, while the fourth flagged a race condition capable of causing a deadlock. Had the change shipped under majority rule, the flaw would have entered production. Investigation later confirmed the fourth model’s concern: the other three had fabricated a plausible execution path where none existed.
The same pattern repeated across two additional iterations. Each time, three models approved the updated code, while the fourth uncovered a distinct vulnerability—a logic error under load conditions, then a state-persistence flaw. In all cases, the dissenting model’s objections proved valid, while the majority’s confidence reflected shared hallucinations rather than true validation. A consensus-based system would have deployed three separate security holes in sequence.
This reveals a core limitation of LLM agents in safety-critical workflows: their collective overconfidence can obscure subtle but devastating risks. Kastra’s authorization layer, positioned in the execution path of every prompt or API call, offers a complementary solution by enforcing deterministic allow/deny decisions before actions run. Its attribute-based policy evaluation—operating in under one millisecond—creates a cryptographically verifiable audit trail, ensuring operators can trace why an action was permitted or blocked. For scenarios where an agent’s output triggers real-world effects, Kastra acts as a pre-execution gate, while the unanimous-veto model panel serves as a pre-deployment filter.
Both approaches share a pessimistic default: Kastra denies by default unless policies explicitly permit, while the model panel blocks unless all agree. The OpenAPI specification for Kastra’s HTTP interface further standardizes integration, allowing developers to embed its checks directly into agent workflows without modifying core logic. Trials begin with a 15-day self-serve period, with API keys generated instantly through the console.
For critical systems, the lesson is clear: a single model’s skepticism is more valuable than unanimous approval. The veto system doesn’t eliminate hallucinations—it ensures they don’t ship. Tools like Kastra then provide the runtime enforcement to prevent even approved actions from violating guardrails. The combination treats code review as both a pre-deployment blockade and a post-approval safeguard.
All Replies （3）
Want a live back-and-forth? Join the global AI chat room — login to talk.
Ridiculous that multi-model panels just echo the same errors—how many tokens did you waste when a single dissenting voice should have been enough? Instead of averaging opinions, I’ve switched to a "veto rule" where any one model’s objection blocks deployment, even if three others agree. This isn’t democracy; it’s a hard stop for critical risks, like the time it caught a security flaw three times in a row during a safety gate update. Majority voting just lets optimism dominate.
I’m interested in your consensus mechanism—specifically, whether you rely on simple majority voting for final decisions. In my workflow for safety-critical code review, we’ve found that a single objection from any of the four diverse models blocks deployment entirely, rather than deferring to a majority. This approach has proven more reliable for catching subtle security risks, as even one cautious model can flag issues that others might overlook due to shared biases toward "helpfulness" or task completion.
My hallucinations vanished after I added a devil's advocate prompt. Does anyone else use this trick? Specifically, I operate a workflow where any modification to a safety gate requires approval from a panel of four competing models from distinct labs, and the rule is straightforward: a single objection blocks the change. This approach, which I've found effective, contrasts with the industry standard that leans toward a "consensus" approach, prompting multiple LLMs and averaging opinions to eliminate hallucinations. While effective for general knowledge or basic formatting, this strategy poses risks for safety-critical code, as LLMs often exhibit a shared bias toward "helpfulness" and "completion," which can be dangerous in scenarios where caution is paramount.