7M parameter models are outperforming GPT on ARC and it's kind
If you haven't been following the ARC (Abstraction and Reasoning Corpus) drama, here is the deal: it's a benchmark designed by Francois Chollet to test fluid intelligence. It uses visual grid transformations where you have to infer a rule from a few examples and apply it to a new one. It's not about memorizing text or retrieving facts; it's about pure, raw pattern recognition.
For a long time, even the heavyweights like Gemini or Claude struggled to move the needle on this. But then the data started coming in from two different directions that suggest "bigger" isn't always "smarter."
The Tiny Recursive Model vs. The Giants
Samsung's Tiny Recursive Model (TRM) is a complete outlier. We are talking about a model that is roughly 0.01% the size of GPT-4. It only has 7 million parameters and is just two layers deep. Yet, on the ARC-AGI-2 benchmark, its performance is actually making the frontier models look sluggish.
Here is how the numbers look for ARC-AGI-2:
Think about that for a second. A model that could easily fit on a thumb drive is outperforming systems that cost hundreds of millions of dollars to train. It suggests that for specific types of reasoning, architecture and recursive logic matter way more than just raw parameter count.
The Solo Dev Who Proved It
It wasn't just a corporate lab either. A solo developer, Mithil Vakde, managed to train a small transformer from scratch on a single RTX 5090 in about 90 minutes. He hit 44% on the ARC-AGI-1 benchmark.
This wasn't some massive pre-training run on the entire internet. He used a very specific AI workflow involving:
1. Test-time training
2. 3D RoPE embeddings
3. Aggressive data augmentation (using color and dihedral permutations)
His point is crucial: this isn't a standard LLM. It's a specialized autoregressive transformer built for sample efficiency. When he says sample efficiency is one of the biggest unsolved problems in AI, he's hitting the nail on the head. We are currently building these massive "digital encyclopedias" that are incredibly smart at reciting facts but surprisingly brittle when they encounter a logic puzzle they haven't seen before.
Why this matters for the future
We are seeing a massive surge in interest for projects like minimind, which lets you train a 64M-parameter model for about three dollars in two hours. The GitHub stars are exploding because people are realizing that the "frontier" might not just be a race to the top of the compute mountain.
There is a huge gap between "tiny" and "frontier," and the winner of the next era of AI might not be the one with the most GPUs, but the one who figures out how to bake reasoning into a much smaller, more efficient package. If we can get GPT-level reasoning into a model that runs locally on a phone or a laptop without needing a massive server farm, the deployment landscape changes overnight.