Why a high-school dropout is currently outperforming PhDs in AI

Mathematics has always been the final boss for LLMs because they struggle with the "hallucination" of logic—they predict the next token rather than actually reasoning through a proof. However, the recent shift toward integrating reinforcement learning with formal verification is finally cracking this. The most interesting part isn't just the technical leap, but the fact that some of the most impactful breakthroughs are coming from unconventional thinkers who approach prompt engineering and logic from a first-principles perspective rather than academic dogma.

The core of the problem is that natural language is too ambiguous for complex math. When an AI tries to solve a hard geometry problem in English, it often makes a "leap of faith" that looks correct but is logically bankrupt. The breakthrough comes when we force the AI to work within a formal language—like Lean or Isabelle—where every single step must be mathematically verified by a kernel. If the kernel says the step is wrong, the AI can't just "convince" the user it's right; it has to backtrack and try a different logical path.

For those trying to build a similar AI workflow for technical tasks, here is a practical tutorial on how to implement a basic verification loop:

1. Define the Formal Target: Instead of asking for a solution in text, require the output in a formal language. For example, use a Lean 4 snippet to define the theorem you are trying to prove.
2. Implement the Generator: Use a high-reasoning model (like Claude 3.5 Sonnet or GPT-4o) to generate a candidate proof.
3. The Verification Gate: Pass that output through a compiler or formal verifier.

# Example of running a Lean file to check for errors
lean .my_proof.lean

4. Error Feedback Loop: If the compiler returns an error, feed the exact error message back into the LLM. This creates a self-correcting loop where the AI learns from the formal constraints of the math, not just the patterns in its training data.

• Standard LLM approach: Probability-based guessing, high hallucination rate in multi-step logic, looks plausible but often fails.
• Formal Verification approach: Deterministic correctness, zero tolerance for logical gaps, slower generation but 100% reliability once verified.

This shift proves that the next era of LLM agents isn't about bigger datasets, but about better "guardrails" for reasoning. By combining the creative intuition of a model with the rigid laws of formal mathematics, we are seeing AI solve problems that were previously thought to require human-level insight. It's a reminder that the most effective prompt engineering often comes from people who aren't afraid to break the "standard" way of doing things.