# Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models

arXiv: https://arxiv.org/abs/2502.20332 · HTML: https://mlsanity.com/paper/2502.20332/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 31 of its 52 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Faith and Fate: Limits of Transformers on Compositionality](https://mlsanity.com/paper/2305.18654/) (2023)
- [Function Vectors in Large Language Models](https://mlsanity.com/paper/2310.15213/) (2023)
- [Direct and Indirect Effects](https://mlsanity.com/paper/1301.2300/) (2013)
- [Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks](https://mlsanity.com/paper/2307.02477/) (2023)
- [Emergent Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2212.09196/) (2022)
- [Large Language Models as General Pattern Machines](https://mlsanity.com/paper/2307.04721/) (2023)
- [Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve](https://mlsanity.com/paper/2309.13638/) (2023)
- [Language models show human-like content effects on reasoning tasks](https://mlsanity.com/paper/2207.07051/) (2022)
- [From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought](https://mlsanity.com/paper/2306.12672/) (2023)
- [How do Language Models Bind Entities in Context?](https://mlsanity.com/paper/2310.17191/) (2023)
- [In-Context Language Learning: Architectures and Algorithms](https://mlsanity.com/paper/2401.12973/) (2024)
- [Evaluating Cognitive Maps and Planning in Large Language Models with CogEval](https://mlsanity.com/paper/2309.15129/) (2023)
- [Which Attention Heads Matter for In-Context Learning?](https://mlsanity.com/paper/2502.14010/) (2025)
- [A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Task](https://mlsanity.com/paper/2402.11917/) (2024)
- [Emergent Symbols through Binding in External Memory](https://mlsanity.com/paper/2012.14601/) (2020)
- [Evaluating the Robustness of Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2411.14215/) (2024)
- [The Relational Bottleneck as an Inductive Bias for Efficient Abstraction](https://mlsanity.com/paper/2309.06629/) (2023)
- [On Neural Architecture Inductive Biases for Relational Tasks](https://mlsanity.com/paper/2206.05056/) (2022)
- [Evidence from counterfactual tasks supports emergent analogical reasoning in large language models](https://mlsanity.com/paper/2404.13070/) (2024)
- [Abstractors and relational cross-attention: An inductive bias for explicit relational reasoning in Transformers](https://mlsanity.com/paper/2304.00195/) (2023)
- [An explainable transformer circuit for compositional generalization](https://mlsanity.com/paper/2502.15801/) (2025)
- [Emergent Symbol-like Number Variables in Artificial Neural Networks](https://mlsanity.com/paper/2501.06141/) (2025)

## Built on by (most cited first, 30 of 31)

- [Language Models use Lookbacks to Track Beliefs](https://mlsanity.com/paper/2505.14685/) (2025)
- [Emergent Abilities in Large Language Models: A Survey](https://mlsanity.com/paper/2503.05788/) (2025)
- [The Dual-Route Model of Induction](https://mlsanity.com/paper/2504.03022/) (2025)
- [Stepwise Guided Policy Optimization: Coloring your Incorrect Reasoning in GRPO](https://mlsanity.com/paper/2505.11595/) (2025)
- [Causal Head Gating: A Framework for Interpreting Roles of Attention Heads in Transformers](https://mlsanity.com/paper/2505.13737/) (2025)
- [Visual symbolic mechanisms: Emergent symbol processing in vision language models](https://mlsanity.com/paper/2506.15871/) (2025)
- [Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learning](https://mlsanity.com/paper/2508.01916/) (2025)
- [Causality neq Invariance: Function and Concept Vectors in LLMs](https://mlsanity.com/paper/2602.22424/) (2026)
- [Whither symbols in the era of advanced neural networks?](https://mlsanity.com/paper/2508.05776/) (2025)
- [LLMs Process Lists With General Filter Heads](https://mlsanity.com/paper/2510.26784/) (2025)
- [The Mechanistic Emergence of Symbol Grounding in Language Models](https://mlsanity.com/paper/2510.13796/) (2025)
- [Learning to Ponder: Adaptive Reasoning in Latent Space](https://mlsanity.com/paper/2509.24238/) (2025)
- [In-Context Algebra](https://mlsanity.com/paper/2512.16902/) (2025)
- [Illusion or Algorithm? Investigating Memorization, Emergence, and Symbolic Processing in In-Context Learning](https://mlsanity.com/paper/2505.11004/) (2025)
- [Understanding In-Context Learning for Nonlinear Regression with Transformers: Attention as Featurizer](https://mlsanity.com/paper/2605.05176/) (2026)
- [A circuit for predicting hierarchical structure in-context in Large Language Models](https://mlsanity.com/paper/2509.21534/) (2025)
- [Understanding the Staged Dynamics of Transformers in Learning Latent Structure](https://mlsanity.com/paper/2511.19328/) (2025)
- [Can Large Language Models Generalize Procedures Across Representations?](https://mlsanity.com/paper/2602.03542/) (2026)
- [Grokking From Abstraction to Intelligence](https://mlsanity.com/paper/2603.29262/) (2026)
- [Mechanisms of Object Localization in Vision-Language Models](https://mlsanity.com/paper/2605.19792/) (2026)
- [Who Says What: Symbolic Trimodal Binding Mechanisms in Audio-Visual LLMs](https://mlsanity.com/paper/2609.31193/) (2026)
- [A Group Theoretic Analysis of the Symmetries Underlying Base Addition and Their Learnability by Neural Networks](https://mlsanity.com/paper/2507.10678/) (2025)
- [Representing Beauty: Towards a Participatory but Objective Latent Aesthetics](https://mlsanity.com/paper/2510.02869/) (2025)
- [In-Context Learning Without Copying](https://mlsanity.com/paper/2511.05743/) (2025)
- [Emergent Structured Representations Support Flexible In-Context Inference in Large Language Models](https://mlsanity.com/paper/2602.07794/) (2026)
- [How Do Language Models Understand Tables? A Mechanistic Analysis of Cell Location](https://mlsanity.com/paper/2602.08548/) (2026)
- [Binding Visual Features Point by Point](https://mlsanity.com/paper/2605.25427/) (2026)
- [HPP: Hierarchical Programmatic Probing for Long Video Understanding by Decoupling Perception and Reasoning](https://mlsanity.com/paper/2606.21734/) (2026)
- [A Pre-Specified Construction-Confirmation Test of Operation-Level Causal Transfer Across Finite Isomorphic Symbolic Domains](https://mlsanity.com/paper/2608.15809/) (2026)
- [Logical subspace in LLMs](https://mlsanity.com/paper/2609.32907/) (2026)

## Abstract (arXiv)

Many recent studies have found evidence for emergent reasoning capabilities in large language models (LLMs), but debate persists concerning the robustness of these capabilities, and the extent to which they depend on structured reasoning mechanisms. To shed light on these issues, we study the internal mechanisms that support abstract reasoning in LLMs. We identify an emergent symbolic architecture that implements abstract reasoning via a series of three computations. In early layers, symbol abstraction heads convert input tokens to abstract variables based on the relations between those tokens. In intermediate layers, symbolic induction heads perform sequence induction over these abstract variables. Finally, in later layers, retrieval heads predict the next token by retrieving the value associated with the predicted abstract variable. These results point toward a resolution of the longstanding debate between symbolic and neural network approaches, suggesting that emergent reasoning in neural networks depends on the emergence of symbolic mechanisms.