A 150M parameter model hitting 29.5% on ARC-AGI-1 is insane

Getting a 29.5% score on ARC-AGI-1 is usually the domain of massive models or incredibly expensive compute setups, but this new paper from the Pathway team shows a 150M parameter model doing it for about $0.0007 per task. The most interesting part isn't even the score—it's the fact that this isn't a transformer.

We're looking at a recurrent latent reasoning setup. Instead of the standard token-by-token prediction we see in LLMs, this thing basically "thinks" in latent space, cycling through internal states before it ever commits to an answer. It’s essentially a deep dive into how we can achieve complex reasoning without needing a trillion parameters or a massive GPU cluster. Because the model is so tiny, it could practically run on a toaster, yet it sits completely outside the current cost-to-accuracy frontier for the ARC-AGI benchmark.

Why latent reasoning matters

Most of our current AI workflow relies on the transformer architecture, which is great but computationally hungry. This recurrent approach suggests that if the model can iterate on a problem internally—essentially giving itself "time to think" without generating intermediate text—it can solve logic puzzles that usually trip up small models. It's a different way of handling the "system 2" thinking process.

If you're into prompt engineering or building LLM agents, this is a signal that the future might not just be "bigger models," but more efficient internal architectures. When a model this small can punch that far above its weight class, it makes you wonder what happens when this architecture is scaled.

I'm not ready to declare the death of the transformer yet, but if this scales to 1B or 3B parameters, the efficiency gains would be monstrous. We're talking about potential local deployment of high-reasoning models that don't drain your battery or require a $40k H100.

For those who want to dig into the technicals, the paper is available on arXiv:

https://arxiv.org/abs/2608.09888

The trade-off between parameter count and reasoning capability is shifting. While everyone is chasing the next 10-trillion parameter behemoth, these smaller, recurrent setups might actually be the path to real-world AGI because they are actually deployable. It'll be interesting to see if this approach holds up across other benchmarks or if it's specifically tuned for the grid-logic nature of ARC.