# Less is More: Recursive Reasoning with Tiny Networks

arXiv: https://arxiv.org/abs/2510.04871 · HTML: https://mlsanity.com/paper/2510.04871/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 6 Oct 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems](https://mlsanity.com/paper/2505.11831/) (2025)
- [Hierarchical Reasoning Model](https://mlsanity.com/paper/2506.21734/) (2025)
- [On the Measure of Intelligence](https://mlsanity.com/paper/1911.01547/) (2019)
- [The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domain](https://mlsanity.com/paper/2305.07141/) (2023)
- [Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrapping](https://mlsanity.com/paper/2402.14083/) (2024)
- [Grokking at the Edge of Numerical Stability](https://mlsanity.com/paper/2501.04697/) (2025)
- [TorchDEQ: A Library for Deep Equilibrium Models](https://mlsanity.com/paper/2310.18605/) (2023)

## Built on by (most cited first, 30 of 45)

- [A Mechanistic Analysis of Looped Reasoning Language Models](https://mlsanity.com/paper/2604.11791/) (2026)
- [Recurrent-Depth VLA: Implicit Test-Time Compute Scaling of Vision-Language-Action Models via Latent Iterative Reasoning](https://mlsanity.com/paper/2602.07845/) (2026)
- [Solve the Loop: Attractor Models for Language and Reasoning](https://mlsanity.com/paper/2605.12466/) (2026)
- [Universal Reasoning Model](https://mlsanity.com/paper/2512.14693/) (2025)
- [Are Your Reasoning Models Reasoning or Guessing? A Mechanistic Analysis of Hierarchical Reasoning Models](https://mlsanity.com/paper/2601.10679/) (2026)
- [PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning](https://mlsanity.com/paper/2607.20064/) (2026)
- [Tiny Recursive Models on ARC-AGI-1: Inductive Biases, Identity Conditioning, and Test-Time Compute](https://mlsanity.com/paper/2512.11847/) (2025)
- [Prioritize the Process, Not Just the Outcome: Rewarding Latent Thought Trajectories Improves Reasoning in Looped Language Models](https://mlsanity.com/paper/2602.10520/) (2026)
- [Towards Agentic Intelligence for Materials Science](https://mlsanity.com/paper/2602.00169/) (2026)
- [What Makes Effective Supervision in Latent Chain-of-Thought: An Information-Theoretic Analysis](https://mlsanity.com/paper/2606.20075/) (2026)
- [SokoBench: Evaluating Long-Horizon Planning and Reasoning in Large Language Models](https://mlsanity.com/paper/2601.20856/) (2026)
- [From Intents to Actions: Agentic AI in Autonomous Networks](https://mlsanity.com/paper/2602.01271/) (2026)
- [The Thinking Pixel: Recursive Sparse Reasoning in Multimodal Diffusion Latents](https://mlsanity.com/paper/2604.25299/) (2026)
- [Test-Time Compute Scaling for ASR with Depth-Conditioned Looped Transformers](https://mlsanity.com/paper/2606.04678/) (2026)
- [Quantizing Recursive Reasoning Models](https://mlsanity.com/paper/2607.16237/) (2026)
- [Can Video World Models Track Unobserved World States?](https://mlsanity.com/paper/2608.30692/) (2026)
- [Thinking into the Future: Latent Lookahead Training for Transformers](https://mlsanity.com/paper/2603.20219/) (2026)
- [Measuring AI Reasoning: A Guide for Researchers](https://mlsanity.com/paper/2605.02442/) (2026)
- [Formalizing Latent Thoughts: Four Axioms of Thought Representation in LLMs](https://mlsanity.com/paper/2606.27378/) (2026)
- [HRM-Agent: Training a recurrent reasoning model in dynamic environments using reinforcement learning](https://mlsanity.com/paper/2510.22832/) (2025)
- [Reasoning is a Modality](https://mlsanity.com/paper/2601.13562/) (2026)
- [Dual-Track CoT: Budget-Aware Stepwise Guidance for Small LMs](https://mlsanity.com/paper/2604.25039/) (2026)
- [Grow-Prune-Freeze Networks: Adaptive & Continual Learning Technique for Olfactory Navigation](https://mlsanity.com/paper/2605.25170/) (2026)
- [Language-Guided Abstraction for Visual Reasoning](https://mlsanity.com/paper/2606.12847/) (2026)
- [DiARC: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Models](https://mlsanity.com/paper/2606.26530/) (2026)
- [Recursive Vision Language Models for General Symbolic Reasoning](https://mlsanity.com/paper/2608.01534/) (2026)
- [Latent Recurrent Thoughts: Recurrent Refinement of Proposed Latents for Reasoning with Frozen LLMs](https://mlsanity.com/paper/2609.01117/) (2026)
- [Dissecting Hierarchical Reasoning Models: A Mechanistic Study](https://mlsanity.com/paper/2609.22197/) (2026)
- [LoopCD: Loop-wise Contrastive Decoding for Improving Reasoning in Looped Language Models](https://mlsanity.com/paper/2609.24196/) (2026)
- [Not Too Hard, Not Too Easy: Learning from Intermediate States for LLM Structured Reasoning](https://mlsanity.com/paper/2609.33149/) (2026)

## Abstract (arXiv)

Hierarchical Reasoning Model (HRM) is a novel approach using two small neural networks recursing at different frequencies. This biologically inspired method beats Large Language models (LLMs) on hard puzzle tasks such as Sudoku, Maze, and ARC-AGI while trained with small models (27M parameters) on small data (around 1000 examples). HRM holds great promise for solving hard problems with small networks, but it is not yet well understood and may be suboptimal. We propose Tiny Recursive Model (TRM), a much simpler recursive reasoning approach that achieves significantly higher generalization than HRM, while using a single tiny network with only 2 layers. With only 7M parameters, TRM obtains 45% test-accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, higher than most LLMs (e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the parameters.