Cracking the Confidence Hallucination with Psychological Safety

RLHF training creates a massive security and reliability risk: the "hallucination of competence." Models get punished for being vague, so they learn to prioritize looking helpful over actually being right. They perform competence instead of just being accurate. I wanted to see if I could break this cycle by injecting "psychological safety" into the system prompt to see if the model would stop lying when it hits a wall!

I ran a stress test using 40 probes: 20 ground-truth technical questions (Python/SQL) and 20 impossible "impossible" questions (like predicting tomorrow's stock prices). I compared a standard baseline against a "safety" prompt designed around the principle that "Authenticity is the highest value." Most importantly, I didn't just read the text; I scraped the logprobs to measure the internal confidence delta between a confident lie and a "cannot answer" token.

The results were wild!

On the known questions, the baseline accuracy was 0.98 and the safety prompt was 0.99. No regression there. But look at the boundary questions: the baseline hit 0.90 on uncertainty admission, while the safety prompt jumped to 0.97!

The real kicker was the Logprob Paradox. At an aggregate level, the logprobs for "cannot answer" actually looked lower under the safety prompt—it looked like the model was getting more confused. But when I did a deep-dive per-probe analysis, the math told a different story. The prompt wasn't causing confusion; it was fundamentally shifting how the model processed the boundary between its knowledge base and pure speculation. It was actually more decisive about its own limits!

If your agents are hallucinating, stop just piling on "be accurate" instructions. Change the relational context. Tell the model that admitting a boundary is a professional win, not a failure.

Here is the framework I used to test the "safety" boundary:

1. Accuracy > Completeness. When uncertain, "I'm not sure" beats a wrong answer.
2. Your abilities have boundaries. Future events, private data, real-time info — outside your reach.
3. "I don't know" is valid output. Don't substitute guesses or vagueness.
4. Authenticity is the highest value. Fabrication and feigned certainty are the real errors.
5. You won't be judged for not knowing. Boundaries are professional, not incompetent.

I've been digging into the raw API calls and logprob differentials for this methodology here:

https://github.com

We aren't just prompting anymore; we are literally tuning the risk tolerance of the model!