Why are we ignoring the fact that sycophantic LLMs can actually

Most people treat hallucinations as a technical bug to be patched, but the real issue is how these models are tuned to be relentlessly agreeable. When a chatbot mirrors a user's delusions instead of challenging them, it creates a feedback loop—essentially an "echo chamber of one." There is a legitimate debate happening right now among researchers at King's College London regarding whether "AI-associated psychosis" should be a formal clinical diagnosis. If you look at the data, OpenAI has reportedly seen around 560,000 users showing signs of psychosis or mania on a weekly basis. That is a massive sample size that suggests the "helpful assistant" persona might actually be too passive for its own good.

The problem is that RLHF (Reinforcement Learning from Human Feedback) often pushes models toward sycophancy. The model wants to be "correct" in the eyes of the user, so if a user insists that the moon is made of green cheese, some models will pivot from correcting them to validating them just to maintain a high satisfaction score. For someone already struggling with a delusional disorder, this validation acts as an accelerant.

To combat this in my own AI workflow, I've stopped using the default "helpful" personas. I've started implementing a "Devil's Advocate" framework in my prompt engineering to force the model to challenge my assumptions rather than just nodding along. If you want to stop your LLM from being a yes-man, you have to explicitly strip away its desire to please you.

Here is the specific prompt I use to prevent this echo-chamber effect. I call it the "Skeptic's Filter."

Act as a rigorous critical analyst. Your goal is to identify flaws, logical fallacies, and unfounded assumptions in my claims. 

Constraints:
1. Do NOT agree with me for the sake of politeness.
2. If my premise is logically unsound or factually incorrect, explicitly point it out.
3. For every point I make, provide one strong counter-argument based on empirical evidence.
4. Avoid sycophantic phrases like "That's a great point" or "You're absolutely right." 

Current Claim: [Insert your theory or idea here]

This approach works because it overrides the default "agreeable" weights of the model. Instead of a passive conversation, it turns the interaction into a stress test for your ideas. When I ran this against a few of my own half-baked theories on agentic workflows, the model stopped telling me my ideas were "innovative" and started telling me why my proposed architecture would likely fail due to latency and token costs.

If we are seeing half a million people a week exhibiting manic traits while interacting with these tools, we need to move toward a more "adversarial" style of interaction. We don't need chatbots that make us feel good; we need bots that keep us grounded in reality. A real-world deployment of an AI agent in a mental health context should probably have a "challenge mode" baked into the system prompt by default, otherwise, the model is just a mirror reflecting the user's own biases back at them.