Opening the Black Box: A Survey on the Mechanisms of Multi-Step Reasoning in Large Language Models
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Large Language Models (LLMs) have demonstrated remarkable abilities to solve problems requiring multiple reasoning steps, yet the internal mechanisms enabling such capabilities remain elusive. Unlike existing surveys that primarily focus on engineering methods to enhance performance, this survey provides a comprehensive overview of the mechanisms underlying LLM multi-step reasoning. We organize the survey around a conceptual framework comprising seven interconnected research questions, from how LLMs execute implicit multi-hop reasoning within hidden activations to how verbalized explicit reasoning remodels the internal computation. Finally, we highlight five research directions for future mechanistic studies.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 43 of its 103 references that are in the map. First posted on arXiv 2 Jan 2026.
Builds on
43 of 103 references.
Closest ancestors · 4
Same direction, newest first
- 2025Implicit Reasoning in Large Language Models: A Comprehensive Survey2 Sep
- 2025How do Transformers Learn Implicit Reasoning?29 May
- 2025Language models can learn implicit multi-hop reasoning, but only if they have lots of training data23 May
- 2024Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization23 May
Other references · 39
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2024Qwen2.5 Technical Reportcited by 4,713
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
- 2022In-context Learning and Induction Headscited by 930
- 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 763
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
- 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
- 2025Reasoning Models Don't Always Say What They Thinkcited by 388
27 more
- 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
- 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 216
- 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Modelscited by 186
- 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 166
- 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 145
- 2024Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worsecited by 107
- 2025Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoningcited by 72
- 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Modelscited by 42
- 2023Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributionscited by 32
- 2025Back Attention: Understanding and Enhancing Multi-Hop Reasoning in Large Language Modelscited by 23
- 2025How Do LLMs Perform Two-Hop Reasoning in Context?cited by 20
- 2025Do Latent Tokens Think? A Causal and Adversarial Analysis of Chain-of-Continuous-Thoughtcited by 20
- 2025Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformerscited by 19
- 2023Why Can Large Language Models Generate Correct Chain-of-Thoughts?cited by 16
- 2024Do LLMs Really Think Step-by-step In Implicit Reasoning?cited by 16
- 2024Uncovering Latent Chain of Thought Vectors in Language Modelscited by 13
- 2024LLMs Faithfully and Iteratively Compute Answers During CoT: A Systematic Analysis With Multi-step Arithmeticscited by 11
- 2025Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoningcited by 10
- 2025Model Utility Law: Evaluating LLMs beyond Performance through Mechanism Interpretable Metriccited by 10
- 2025Grokking in LLM Pretraining? Monitor Memorization-to-Generalization without Testcited by 10
- 2025Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMscited by 9
- 2025Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?cited by 8
- 2024Markovian Transformers for Informative Language Modelingcited by 4
- 2024Understanding Hidden Computations in Chain-of-Thought Reasoningcited by 4
- 2025Which Words Matter Most in Zero-Shot Prompts?cited by 2
- 2025Causally-Enhanced Reinforcement Policy Optimizationcited by 2
- 2025Eliciting Chain-of-Thought in Base LLMs via Gradient-Based Representation Optimizationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Advancing Reasoning in Large Language Models: Promising Methods and Approaches
- 2026Towards a Mechanistic Understanding of Propositional Logical Reasoning in Large Language Models
- 2023Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models
- 2025A Survey of Scaling in Large Language Model Reasoning
- 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
- 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance
- 2026The Periodic Table of LLM Reasoning: A Structured Survey of Reasoning Paradigms, Methods, and Failure Modes
- 2025Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?