# How do Transformers Learn Implicit Reasoning?

arXiv: https://arxiv.org/abs/2505.23653 · HTML: https://mlsanity.com/paper/2505.23653/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 29 May 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 20 of 20)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning](https://mlsanity.com/paper/2504.01296/) (2025)
- [Let's Think Dot by Dot: Hidden Computation in Transformer Language Models](https://mlsanity.com/paper/2404.15758/) (2024)
- [Reasoning Models Can Be Effective Without Thinking](https://mlsanity.com/paper/2504.09858/) (2025)
- [Distilling System 2 into System 1](https://mlsanity.com/paper/2407.06023/) (2024)
- [Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing](https://mlsanity.com/paper/2410.06331/) (2024)
- [Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?](https://mlsanity.com/paper/2411.16679/) (2024)
- [Distributional reasoning in LLMs: Parallel reasoning processes in multi-hop reasoning](https://mlsanity.com/paper/2406.13858/) (2024)
- [Lessons from Studying Two-Hop Latent Reasoning](https://mlsanity.com/paper/2411.16353/) (2024)
- [Is the Reversal Curse a Binding Problem? Uncovering Limitations of Transformers from a Basic Generalization Failure](https://mlsanity.com/paper/2504.01928/) (2025)
- [AtomR: Atomic Operator-Empowered Large Language Models for Heterogeneous Knowledge Reasoning](https://mlsanity.com/paper/2411.16495/) (2024)
- [Enhancing Multi-hop Reasoning through Knowledge Erasure in Large Language Model Editing](https://mlsanity.com/paper/2408.12456/) (2024)
- [CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners](https://mlsanity.com/paper/2503.16356/) (2025)
- [Towards Interpreting Language Models: A Case Study in Multi-Hop Reasoning](https://mlsanity.com/paper/2411.05037/) (2024)
- [Understanding Hidden Computations in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2412.04537/) (2024)
- [Memorizing is Not Enough: Deep Knowledge Injection Through Reasoning](https://mlsanity.com/paper/2504.00472/) (2025)

## Built on by (most cited first, 16 of 16)

- [Implicit Reasoning in Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.02350/) (2025)
- [AlignSAE: Concept-Aligned Sparse Autoencoders](https://mlsanity.com/paper/2512.02004/) (2025)
- [Capabilities and Fundamental Limits of Latent Chain-of-Thought](https://mlsanity.com/paper/2602.01148/) (2026)
- [Farther the Shift, Sparser the Representation: Analyzing OOD Mechanisms in LLMs](https://mlsanity.com/paper/2603.03415/) (2026)
- [DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoning](https://mlsanity.com/paper/2607.00341/) (2026)
- [Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMs](https://mlsanity.com/paper/2510.07697/) (2025)
- [R-Capsule: Compressing High-Level Plans for Efficient Large Language Model Reasoning](https://mlsanity.com/paper/2509.22131/) (2025)
- [Deductive Logic in Language Models: Horizontal vs Vertical Reasoning](https://mlsanity.com/paper/2510.09340/) (2025)
- [Is Grokking Worthwhile? Functional Analysis and Transferability of Generalization Circuits in Transformers](https://mlsanity.com/paper/2601.09049/) (2026)
- [Opening the Black Box: A Survey on the Mechanisms of Multi-Step Reasoning in Large Language Models](https://mlsanity.com/paper/2601.14270/) (2026)
- [The Kinetics of Reasoning: How Chain-of-Thought Shapes Learning in Transformers?](https://mlsanity.com/paper/2510.25791/) (2025)
- [How Do Language Models Understand Tables? A Mechanistic Analysis of Cell Location](https://mlsanity.com/paper/2602.08548/) (2026)
- [The Power of Power Law: Asymmetry Enables Compositional Reasoning](https://mlsanity.com/paper/2604.22951/) (2026)
- [HyperLens: Quantifying Cognitive Effort in LLMs with Fine-grained Confidence Trajectory](https://mlsanity.com/paper/2605.05741/) (2026)
- [Behaviour Is an Incomplete Measure of Reasoning Development: Cross-surface pre-arrival accessibility and the limits of developmental inference in a recurrent-depth reasoner](https://mlsanity.com/paper/2608.16085/) (2026)
- [From Copying to Composition: How Language Models Develop In-Context Learning](https://mlsanity.com/paper/2610.19113/) (2026)

## Abstract (arXiv)

Recent work suggests that large language models (LLMs) can perform multi-hop reasoning implicitly -- producing correct answers without explicitly verbalizing intermediate steps -- but the underlying mechanisms remain poorly understood. In this paper, we study how such implicit reasoning emerges by training transformers from scratch in a controlled symbolic environment. Our analysis reveals a three-stage developmental trajectory: early memorization, followed by in-distribution generalization, and eventually cross-distribution generalization. We find that training with atomic triples is not necessary but accelerates learning, and that second-hop generalization relies on query-level exposure to specific compositional structures. To interpret these behaviors, we introduce two diagnostic tools: cross-query semantic patching, which identifies semantically reusable intermediate representations, and a cosine-based representational lens, which reveals that successful reasoning correlates with the cosine-base clustering in hidden space. This clustering phenomenon in turn provides a coherent explanation for the behavioral dynamics observed across training, linking representational structure to reasoning capability. These findings provide new insights into the interpretability of implicit multi-hop reasoning in LLMs, helping to clarify how complex reasoning processes unfold internally and offering pathways to enhance the transparency of such models.