mlsanityData to 14 Oct 2026

Hierarchical Reasoning Model

arXiv 2506.21734 ↗First posted 26 Jun 2025cs.AI cs.LGcited by 177

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Reasoning, the process of devising and executing complex goal-oriented action sequences, remains a critical challenge in AI. Current large language models (LLMs) primarily employ Chain-of-Thought (CoT) techniques, which suffer from brittle task decomposition, extensive data requirements, and high latency. Inspired by the hierarchical and multi-timescale processing in the human brain, we propose the Hierarchical Reasoning Model (HRM), a novel recurrent architecture that attains significant computational depth while maintaining both training stability and efficiency. HRM executes sequential reasoning tasks in a single forward pass without explicit supervision of the intermediate process, through two interdependent recurrent modules: a high-level module responsible for slow, abstract planning, and a low-level module handling rapid, detailed computations. With only 27 million parameters, HRM achieves exceptional performance on complex reasoning tasks using only 1000 training samples. The model operates without pre-training or CoT data, yet achieves nearly perfect performance on challenging tasks including complex Sudoku puzzles and optimal path finding in large mazes. Furthermore, HRM outperforms much larger models with significantly longer context windows on the Abstraction and Reasoning Corpus (ARC), a key benchmark for measuring artificial general intelligence capabilities. These results underscore HRM's potential as a transformative advancement toward universal computation and general-purpose reasoning systems.

Key facts. Cited by 177 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 51 of its 99 references that are in the map. First posted on arXiv 26 Jun 2025.

Builds on

51 of 99 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach7 Feb
  2. 2025s1: Simple test-time scaling31 Jan
  3. 2024Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces13 Oct
  4. 2023Large Language Model Guided Tree-of-Thought15 May

Other references · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  6. 2020GLU Variants Improve Transformercited by 1,804
  7. 2019Root Mean Square Layer Normalizationcited by 1,480
  8. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  9. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  10. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 719
  11. 2014Neural Turing Machinescited by 644
  12. 2019Deep Equilibrium Modelscited by 615
35 more
  1. 2017Self-Normalizing Neural Networkscited by 598
  2. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
  3. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 295
  4. 2016Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagationcited by 259
  5. 2021A Geometric Analysis of Neural Collapse with Unconstrained Featurescited by 227
  6. 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 195
  7. 2022The Parallelism Tradeoff: Limitations of Log-Precision Transformerscited by 193
  8. 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systemscited by 190
  9. 2022DeepNet: Scaling Transformers to 1,000 Layerscited by 178
  10. 2021PonderNet: Learning to Pondercited by 168
  11. 2021Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Trainingcited by 166
  12. 2025LIMR: Less is More for RL Scalingcited by 139
  13. 2015Neural GPUs Learn Algorithmscited by 121
  14. 2024ARC Prize 2024: Technical Reportcited by 104
  15. 2024Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrappingcited by 90
  16. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  17. 2024Simplifying Deep Temporal Difference Learningcited by 78
  18. 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoningcited by 78
  19. 2024Scaling Exponents Across Parameterizations and Optimizerscited by 76
  20. 2021On Training Implicit Modelscited by 75
  21. 2014A Clockwork RNNcited by 72
  22. 2017Recurrent Relational Networkscited by 72
  23. 2021Stabilizing Equilibrium Models by Jacobian Regularizationcited by 72
  24. 2024Premise Order Matters in Reasoning with Large Language Modelscited by 72
  25. 2022A Logic for Expressing Log-Precision Transformerscited by 67
  26. 2022Deep Equilibrium Optical Flow Estimationcited by 64
  27. 2025Grokking at the Edge of Numerical Stabilitycited by 46
  28. 2025Log-Linear Attentioncited by 40
  29. 2024Learning Iterative Reasoning through Energy Diffusioncited by 36
  30. 2021SHINE: SHaring the INverse Estimate from the forward pass for bi-level optimization and implicit modelscited by 26
  31. 2023Average-Hard Attention Transformers are Constant-Depth Uniform Threshold Circuitscited by 26
  32. 2025Continuous Thought Machinescited by 22
  33. 2025Sudoku-Bench: Evaluating creative reasoning with Sudoku variantscited by 22
  34. 2024Preemptive Answer "Attacks" on Chain-of-Thought Reasoningcited by 19
  35. 2024Transformers meet Neural Algorithmic Reasonerscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 35; 2026: 142.

177 papers.

Sort:

By area:

  1. 2026LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulationcited by 61
  2. 2026Parcae: Scaling Laws For Stable Looped Language Modelscited by 55
  3. 2025Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrencecited by 47
  4. 2025Less is More: Recursive Reasoning with Tiny Networkscited by 45
  5. 2025Emergent Hierarchical Reasoning in LLMs through Reinforcement Learningcited by 42
  6. 2026A Mechanistic Analysis of Looped Reasoning Language Modelscited by 33
  7. 2025Multimodal Chain of Continuous Thought for Latent-Space Reasoning in Vision-Language Modelscited by 32
  8. 2026Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformerscited by 28
  9. 2025ArcMemo: Abstract Reasoning Composition with Lifelong LLM Memorycited by 24
  10. 2026Generative Recursive Reasoningcited by 20
  11. 2026Probabilistic Tiny Recursive Modelcited by 19
  12. 2026Recurrent-Depth VLA: Implicit Test-Time Compute Scaling of Vision-Language-Action Models via Latent Iterative Reasoningcited by 18
38 more
  1. 2025Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Timecited by 16
  2. 2025A Formal Comparison Between Chain of Thought and Latent Thoughtcited by 15
  3. 2026LoopViT: Scaling Visual ARC with Looped Transformerscited by 15
  4. 2026LoopUS: Recasting Pretrained LLMs into Looped Latent Refinement Modelscited by 15
  5. 2026Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language Modelscited by 14
  6. 2025The Serial Scaling Hypothesiscited by 13
  7. 2025Two-Scale Latent Dynamics for Recurrent-Depth Transformerscited by 13
  8. 2025Hierarchical Reasoning Models: Perspectives and Misconceptionscited by 13
  9. 2026Solve the Loop: Attractor Models for Language and Reasoningcited by 13
  10. 2026Depth-Recurrent Attention Mixtures: Giving Latent Reasoning the Attention it Deservescited by 12
  11. 2025K2-Think: A Parameter-Efficient Reasoning Systemcited by 11
  12. 2025Universal Reasoning Modelcited by 11
  13. 2026Are Your Reasoning Models Reasoning or Guessing? A Mechanistic Analysis of Hierarchical Reasoning Modelscited by 10
  14. 2026Stability and Generalization in Looped Transformerscited by 10
  15. 2026Recursive Multi-Agent Systemscited by 10
  16. 2026PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoningcited by 10
  17. 2025Memory-Augmented Transformers: A Systematic Review from Neuroscience Principles to Enhanced Model Architecturescited by 9
  18. 2025Neural cellular automata: applications to biology and beyond classical AIcited by 9
  19. 2025Tiny Recursive Models on ARC-AGI-1: Inductive Biases, Identity Conditioning, and Test-Time Computecited by 9
  20. 2026Are Latent Reasoning Models Easily Interpretable?cited by 9
  21. 2026HRM-Text: Efficient Pretraining Beyond Scalingcited by 9
  22. 2026Thinking with Looped Flowscited by 9
  23. 2026Prioritize the Process, Not Just the Outcome: Rewarding Latent Thought Trajectories Improves Reasoning in Looped Language Modelscited by 8
  24. 2026Loop the Loopies!cited by 8
  25. 2025Predictability Enables Parallelization of Nonlinear State Space Modelscited by 7
  26. 2025Thinking Longer, Not Always Smarter: Evaluating LLM Capabilities in Hierarchical Legal Reasoningcited by 7
  27. 2026Knowledge Graphs are Implicit Reward Models: Path-Derived Signals Enable Compositional Reasoningcited by 7
  28. 2026Tiny Recursive Reasoning with Mamba-2 Attention Hybridcited by 7
  29. 2026From Growing to Looping: A Unified View of Iterative Computation in LLMscited by 7
  30. 2026DeepLoop: Depth Scaling for Looped Transformerscited by 7
  31. 2025Think Visually, Reason Textually: Vision-Language Synergy in ARCcited by 6
  32. 2026Tiny Autoregressive Recursive Modelscited by 6
  33. 2026Spherical Flows for Sampling Categorical Datacited by 6
  34. 2026Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgetscited by 6
  35. 2025AlphaApollo: A System for Deep Agentic Reasoningcited by 5
  36. 2025Test-time Adaptation of Tiny Recursive Modelscited by 5
  37. 2025Latent Reasoning in TRMs is Secretly a Policy Improvement Operatorcited by 5
  38. 2025Adapting Like Humans: A Metacognitive Agent with Test-time Reasoningcited by 5

50 most cited of 177 All 177 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Modeling Hierarchical Thinking in Large Reasoning Models
  2. 2025Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models
  3. 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
  4. 2025Implicit Reasoning in Large Language Models: A Comprehensive Survey
  5. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models
  6. 2025Beyond Memorization: Extending Reasoning Depth with Recurrence, Memory and Test-Time Compute Scaling
  7. 2025ReCAP: Recursive Context-Aware Reasoning and Planning for Large Language Model Agents
  8. 2026Unlocking the Working Memory of Large Language Models for Latent Reasoning