Rule chaining hits 95% accuracy on ARC-AGI-2 tasks

The ARC-AGI-2 benchmark is notoriously difficult because it demands cognitive generalization rather than pattern matching. The approach detailed in arXiv:2609.10654v1 shifts away from raw LLM guessing and instead uses a three-stage fallback hierarchy. It essentially treats the grid problem as a sequence of symbolic transformations—first trying deterministic rules, then pattern composition, and finally structural abstraction—which allows the system to solve 230 out of 240 test tasks.

How the rule-chaining hierarchy actually works

The framework doesn't just throw a prompt at the problem; it runs a sequence of solvers where each stage depends on the failure of the previous one.

1. Deterministic Rule Discovery: This is the first line of defense. It looks for atomic transformations. If a pixel moves from (x,y) to (x+1,y) or changes color based on a fixed rule, this module catches it. It handles the "easy" geometric shifts.
2. Pattern-Composition Engine: If the deterministic check fails, it moves here. This stage looks for blocks. It tries to see if the output is just the input block mirrored, repeated, or merged with another block. It uses spatial heuristics to find these repeating units.
3. Structural Abstraction Layer: This is the final fallback for the hardest tasks. It attempts to find hierarchical relationships—like a small grid being nested inside a larger grid—and infers rules based on those nested relationships.

Performance and validation numbers

The data shows this isn't just a theoretical improvement. The system was validated across three different sets:

• Internal training: Passed 995 out of 1000 tasks.
• Evaluation set: Solved 105 out of 120 tasks.
• ARC-AGI-2 Test set: Solved 230 out of 240 tasks.

Hitting over 95% accuracy on the test set is significant because it proves that rule chaining is more robust than relying on the latent space of a transformer. By forcing the model to generate a "reasoning trace" at each stage, the process becomes interpretable. You can see exactly where the logic broke—whether it failed at the atomic level or couldn't find the structural abstraction.

When this approach fails

While the numbers are high, this framework relies heavily on the "atomic" definitions in the first stage. If the transformation doesn't fit into the predefined categories of geometric, color, or object-based analysis, the system is forced into the more computationally expensive abstraction layer. This suggests that the system's ceiling is still tied to how we define those initial symbolic rules.

The lack of task-specific tuning is the biggest win here. Because it uses a general fallback hierarchy, it doesn't need to be "told" what kind of puzzle it's solving; the framework discovers the rule type through the failure of the previous stage.