mlsanityData to 14 Oct 2026

How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding

arXiv 2507.22928 ↗First posted 24 Jul 2025cs.CL cs.AIcited by 19

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 1 more
Abstract (arXiv)

Chain-of-thought (CoT) prompting boosts Large Language Models accuracy on multi-step tasks, yet whether the generated "thoughts" reflect the true internal reasoning process is unresolved. We present the first feature-level causal study of CoT faithfulness. Combining sparse autoencoders with activation patching, we extract monosemantic features from Pythia-70M and Pythia-2.8B while they tackle GSM8K math problems under CoT and plain (noCoT) prompting. Swapping a small set of CoT-reasoning features into a noCoT run raises answer log-probabilities significantly in the 2.8B model, but has no reliable effect in 70M, revealing a clear scale threshold. CoT also leads to significantly higher activation sparsity and feature interpretability scores in the larger model, signalling more modular internal computation. For example, the model's confidence in generating correct answers improves from 1.2 to 4.3. We introduce patch-curves and random-feature patching baselines, showing that useful CoT information is not only present in the top-K patches but widely distributed. Overall, our results indicate that CoT can induce more interpretable internal structures in high-capacity LLMs, validating its role as a structured prompting method.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 40 references that are in the map. First posted on arXiv 24 Jul 2025.

Builds on

28 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Faithful Logical Reasoning via Symbolic Chain-of-Thought28 May
  2. 2024Dissociation of Faithful and Unfaithful Reasoning in LLMs23 May
  3. 2024How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning28 Feb
  4. 2024Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning21 Feb

Other references · 24

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2022Toy Models of Superpositioncited by 971
  4. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  5. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  6. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  7. 2024How to use and interpret activation patchingcited by 214
  8. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  9. 2023Localizing Model Behavior with Path Patchingcited by 195
  10. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 143
  11. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  12. 2024Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Modelscited by 105
12 more
  1. 2023Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patchingcited by 80
  2. 2025Walk the Talk? Measuring the Faithfulness of Large Language Model Explanationscited by 45
  3. 2024The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Modelscited by 29
  4. 2024FaithLM: Towards Faithful Explanations for Large Language Modelscited by 19
  5. 2024A Reply to Makelov et al. (2023)'s "Interpretability Illusion" Argumentscited by 17
  6. 2024Analyzing (In)Abilities of SAEs via Formal Languagescited by 17
  7. 2024Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Modelscited by 15
  8. 2024Evaluating Sparse Autoencoders on Targeted Concept Erasure Taskscited by 15
  9. 2024Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Modelscited by 14
  10. 2024Probing Language Models on Their Knowledge Sourcecited by 11
  11. 2024Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic Approachcited by 6
  12. 2025Tokenized SAEs: Disentangling SAE Reconstructionscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 15.

19 papers.

Sort:

By area:

  1. 2026Do Sparse Autoencoders Identify Reasoning Features in Language Models?cited by 12
  2. 2026Are Your Reasoning Models Reasoning or Guessing? A Mechanistic Analysis of Hierarchical Reasoning Modelscited by 10
  3. 2026Good Arguments Against the People Pleasers: How Reasoning Mitigates (Yet Masks) LLM Sycophancycited by 8
  4. 2026Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answerscited by 6
  5. 2025State over Tokens: Characterizing the Role of Reasoning Tokenscited by 5
  6. 2025Typed Chain-of-Thought: A Curry-Howard Framework for Verifying LLM Reasoningcited by 3
  7. 2025Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanationscited by 2
  8. 2026The Tell-Tale Norm: ℓ2 Magnitude as a Signal for Reasoning Dynamics in Large Language Modelscited by 2
  9. 2026Is my model "mind blurting"? Interpreting the dynamics of reasoning tokens with Recurrence Quantification Analysis (RQA)cited by 1
  10. 2025GraphGhost: Tracing Structures Behind Large Language Modelscited by 0
  11. 2026Characterize Then Distill: Mechanistic Reasoning in Large Output Spacescited by 0
  12. 2026From Plausible to Actionable: A Position on LLM Self-Explanationscited by 0
7 more
  1. 2026Probing the Origins of Reasoning Performance: Representational Quality for Mathematical Problem-Solving in RL vs. SFT Fine-Tuned Modelscited by 0
  2. 2026Thinking vs. NoThinking: Towards Interpreting Reasoning Mechanisms of Large Language Models via Sparse Autoencoderscited by 0
  3. 2026Making Clinical Language Models Auditable: Concept-Guided Fine-Tuning for Robust Predictioncited by 0
  4. 2026StateSwap: Probing Support-Elimination Hidden States in Multiple-Choice Questionscited by 0
  5. 2026Taming CoT Obfuscation in VLMs: From Mechanistic Evidence to Activation Enforcementcited by 0
  6. 2026From Decorative to Load-Bearing: Task Difficulty Shapes the Causal Role of Chain-of-Thoughtcited by 0
  7. 2026Are Stated Reasoning Steps Causally Load-Bearing?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models
  2. 2026Mechanistic Interpretability of Chain-of-Thought Reasoning via Sequential Activation Patching
  3. 2025How Chain-of-Thought Works? Tracing Information Flow from Decoding, Projection, and Activation
  4. 2024Chain-of-Thought in Large Language Models: Decoding, Projection, and Activation
  5. 2026Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations
  6. 2026When Chain-of-Thought Fails, the Solution Hides in the Hidden States
  7. 2025The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning
  8. 2024Unveiling the Statistical Foundations of Chain-of-Thought Prompting Methods