mlsanityData to 14 Oct 2026

A Mechanistic Interpretation of Arithmetic Reasoning in Language Models using Causal Mediation Analysis

arXiv 2305.15054 ↗First posted 24 May 2023cs.CL cs.LGcited by 89

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Mathematical reasoning in large language models (LMs) has garnered significant attention in recent work, but there is a limited understanding of how these models process and store information related to arithmetic tasks within their architecture. In order to improve our understanding of this aspect of language models, we present a mechanistic interpretation of Transformer-based LMs on arithmetic questions using a causal mediation analysis framework. By intervening on the activations of specific model components and measuring the resulting changes in predicted probabilities, we identify the subset of parameters responsible for specific predictions. This provides insights into how information related to arithmetic is processed by LMs. Our experimental results indicate that LMs process the input by transmitting the information relevant to the query from mid-sequence early layers to the final token using the attention mechanism. Then, this information is processed by a set of MLP modules, which generate result-related information that is incorporated into the residual stream. To assess the specificity of the observed activation dynamics, we compare the effects of different model components on arithmetic queries with other tasks, including number retrieval from prompts and factual knowledge questions.

Key facts. Cited by 89 papers in the map (data through 14 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 36 of its 51 references that are in the map. First posted on arXiv 24 May 2023.

Builds on

36 of 51 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models21 Oct
  2. 2022Solving Quantitative Reasoning Problems with Language Models29 Jun
  3. 2022NumGLUE: A Suite of Fundamental yet Challenging Mathematical Reasoning Tasks12 Apr

Other references · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2016Layer Normalizationcited by 5,669
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  9. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  10. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  11. 2019Language Models as Knowledge Bases?cited by 1,936
  12. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
21 more
  1. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  2. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  3. 2022In-context Learning and Induction Headscited by 964
  4. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  5. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  6. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  7. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  8. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  9. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  10. 2021Causal Abstractions of Neural Networkscited by 422
  11. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  12. 2013Direct and Indirect Effectscited by 280
  13. 2015Visualizing and Understanding Recurrent Networkscited by 160
  14. 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 156
  15. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
  16. 2023Jump to Conclusions: Short-Cutting Transformers With Linear Transformationscited by 90
  17. 2023Goat: Fine-tuned LLaMA Outperforms GPT-4 on Arithmetic Taskscited by 84
  18. 2021On the Pitfalls of Analyzing Individual Neurons in Language Modelscited by 62
  19. 2021Measuring and Improving BERT's Mathematical Abilities by Predicting the Order of Reasoningcited by 23
  20. 2021Investigating Numeracy Learning Ability of a Text-to-Text Transfer Modelcited by 13
  21. 2021Masked Measurement Prediction: Learning to Jointly Predict Quantities and Units from Textual Contextcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 33; 2025: 27; 2026: 27.

89 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  2. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  3. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  4. 2024How to use and interpret activation patchingcited by 214
  5. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  6. 2024Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanismscited by 149
  7. 2024Towards Interpreting Visual Information Processing in Vision-Language Modelscited by 122
  8. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Modelscited by 118
  9. 2024Tokenization counts: the impact of tokenization on arithmetic in frontier LLMscited by 115
  10. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
  11. 2024Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristicscited by 97
  12. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 85
38 more
  1. 2025A Survey on Latent Reasoningcited by 81
  2. 2024On the Role of Attention Heads in Large Language Model Safetycited by 72
  3. 2024Attention Heads of Large Language Models: A Surveycited by 67
  4. 2024Confidence Regulation Neurons in Language Modelscited by 57
  5. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 56
  6. 2026When Models Manipulate Manifolds: The Geometry of a Counting Taskcited by 46
  7. 2024Memory3: Language Modeling with Explicit Memorycited by 40
  8. 2024Answer, Assemble, Ace: Understanding How LMs Answer Multiple Choice Questionscited by 35
  9. 2025Detection and Mitigation of Hallucination in Large Reasoning Models: A Mechanistic Perspectivecited by 34
  10. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
  11. 2026Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorcited by 26
  12. 2024Optimal ablation for interpretabilitycited by 25
  13. 2025Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughtscited by 24
  14. 2024Efficient Automated Circuit Discovery in Transformers using Contextual Decompositioncited by 23
  15. 2025Back Attention: Understanding and Enhancing Multi-Hop Reasoning in Large Language Modelscited by 23
  16. 2026Spherical Steering: Geometry-Aware Activation Rotation for Language Modelscited by 21
  17. 2025Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysiscited by 20
  18. 2024Transformers Struggle to Learn to Searchcited by 19
  19. 2024Accelerating Multimodal Large Language Models via Dynamic Visual-Token Exit and the Empirical Findingscited by 15
  20. 2025What is a Number, That a Large Language Model May Know It?cited by 15
  21. 2023Arithmetic with Language Models: from Memorization to Computationcited by 12
  22. 2024Looking Beyond The Top-1: Transformers Determine Top Tokens In Ordercited by 12
  23. 2024Language Models are Symbolic Learners in Arithmeticcited by 11
  24. 2024Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answeringcited by 11
  25. 2026Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal Structurecited by 11
  26. 2024Understanding Synthetic Context Extension via Retrieval Headscited by 10
  27. 2025Compositional Generalization from Learned Skills via CoT Training: A Theoretical and Structural Analysis for Reasoningcited by 10
  28. 2025HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworkscited by 9
  29. 2025Understanding In-context Learning of Addition via Activation Subspacescited by 8
  30. 2025Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learningcited by 8
  31. 2023Conditions for Length Generalization in Learning Reasoning Skillscited by 7
  32. 2024InversionView: A General-Purpose Method for Reading Information from Neural Activationscited by 7
  33. 2024Adaptive Circuit Behavior and Generalization in Mechanistic Interpretabilitycited by 7
  34. 2025Leveraging the true depth of LLMscited by 7
  35. 2024Transformer Normalisation Layers and the Independence of Semantic Subspacescited by 6
  36. 2026Eliciting Numerical Predictive Distributions of LLMs Without Autoregressioncited by 6
  37. 2025Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Modelscited by 5
  38. 2025Correcting Gradient-Based Circuit Localization via Interaction-Aware Backpropagationcited by 5

50 most cited of 89 All 89 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges
  2. 2023Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning
  3. 2024Can Large Language Models put 2 and 2 together? Probing for Entailed Arithmetical Relationships
  4. 2022Reflection of Thought: Inversely Eliciting Numerical Reasoning in Language Models via Solving Linear Systems
  5. 2025Large Language Models and Mathematical Reasoning Failures
  6. 2024Language Models Encode the Value of Numbers Linearly
  7. 2025A Survey on Large Language Models for Mathematical Reasoning
  8. 2026Mechanistic Interpretability of Large-Scale Counting in LLMs through a System-2 Strategy