# How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding

arXiv: https://arxiv.org/abs/2507.22928 · HTML: https://mlsanity.com/paper/2507.22928/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 28 of its 40 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)
- Mechanistic interpretability / [Steering and interpreting LLM reasoning](https://mlsanity.com/mechinterp/steering-and-interpreting-llm-reasoning/)

## Builds on (most cited first, 28 of 28)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [How to use and interpret activation patching](https://mlsanity.com/paper/2404.15255/) (2024)
- [Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability](https://mlsanity.com/paper/2301.04709/) (2023)
- [Localizing Model Behavior with Path Patching](https://mlsanity.com/paper/2304.05969/) (2023)
- [Multi-Step Reasoning with Large Language Models, a Survey](https://mlsanity.com/paper/2407.11511/) (2024)
- [Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning](https://mlsanity.com/paper/2402.13950/) (2024)
- [Faithfulness Tests for Natural Language Explanations](https://mlsanity.com/paper/2305.18029/) (2023)
- [Faithfulness vs. Plausibility: On the (Un)Reliability of Explanations from Large Language Models](https://mlsanity.com/paper/2402.04614/) (2024)
- [Faithful Logical Reasoning via Symbolic Chain-of-Thought](https://mlsanity.com/paper/2405.18357/) (2024)
- [Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching](https://mlsanity.com/paper/2311.17030/) (2023)
- [How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning](https://mlsanity.com/paper/2402.18312/) (2024)
- [Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations](https://mlsanity.com/paper/2504.14150/) (2025)
- [The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Models](https://mlsanity.com/paper/2404.03189/) (2024)
- [Dissociation of Faithful and Unfaithful Reasoning in LLMs](https://mlsanity.com/paper/2405.15092/) (2024)
- [FaithLM: Towards Faithful Explanations for Large Language Models](https://mlsanity.com/paper/2402.04678/) (2024)
- [A Reply to Makelov et al. (2023)'s "Interpretability Illusion" Arguments](https://mlsanity.com/paper/2401.12631/) (2024)
- [Analyzing (In)Abilities of SAEs via Formal Languages](https://mlsanity.com/paper/2410.11767/) (2024)
- [Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Models](https://mlsanity.com/paper/2410.01280/) (2024)
- [Evaluating Sparse Autoencoders on Targeted Concept Erasure Tasks](https://mlsanity.com/paper/2411.18895/) (2024)
- [Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models](https://mlsanity.com/paper/2410.14155/) (2024)
- [Probing Language Models on Their Knowledge Source](https://mlsanity.com/paper/2410.05817/) (2024)
- [Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic Approach](https://mlsanity.com/paper/2410.21779/) (2024)
- [Tokenized SAEs: Disentangling SAE Reconstructions](https://mlsanity.com/paper/2502.17332/) (2025)

## Built on by (most cited first, 19 of 19)

- [Do Sparse Autoencoders Identify Reasoning Features in Language Models?](https://mlsanity.com/paper/2601.05679/) (2026)
- [Are Your Reasoning Models Reasoning or Guessing? A Mechanistic Analysis of Hierarchical Reasoning Models](https://mlsanity.com/paper/2601.10679/) (2026)
- [Good Arguments Against the People Pleasers: How Reasoning Mitigates (Yet Masks) LLM Sycophancy](https://mlsanity.com/paper/2603.16643/) (2026)
- [Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers](https://mlsanity.com/paper/2603.01437/) (2026)
- [State over Tokens: Characterizing the Role of Reasoning Tokens](https://mlsanity.com/paper/2512.12777/) (2025)
- [Typed Chain-of-Thought: A Curry-Howard Framework for Verifying LLM Reasoning](https://mlsanity.com/paper/2510.01069/) (2025)
- [Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations](https://mlsanity.com/paper/2510.17256/) (2025)
- [The Tell-Tale Norm: ℓ2 Magnitude as a Signal for Reasoning Dynamics in Large Language Models](https://mlsanity.com/paper/2606.06188/) (2026)
- [Is my model "mind blurting"? Interpreting the dynamics of reasoning tokens with Recurrence Quantification Analysis (RQA)](https://mlsanity.com/paper/2602.06266/) (2026)
- [GraphGhost: Tracing Structures Behind Large Language Models](https://mlsanity.com/paper/2510.08613/) (2025)
- [Characterize Then Distill: Mechanistic Reasoning in Large Output Spaces](https://mlsanity.com/paper/2606.06840/) (2026)
- [From Plausible to Actionable: A Position on LLM Self-Explanations](https://mlsanity.com/paper/2607.15957/) (2026)
- [Probing the Origins of Reasoning Performance: Representational Quality for Mathematical Problem-Solving in RL vs. SFT Fine-Tuned Models](https://mlsanity.com/paper/2607.26119/) (2026)
- [Thinking vs. NoThinking: Towards Interpreting Reasoning Mechanisms of Large Language Models via Sparse Autoencoders](https://mlsanity.com/paper/2608.08168/) (2026)
- [Making Clinical Language Models Auditable: Concept-Guided Fine-Tuning for Robust Prediction](https://mlsanity.com/paper/2608.27397/) (2026)
- [StateSwap: Probing Support-Elimination Hidden States in Multiple-Choice Questions](https://mlsanity.com/paper/2609.01081/) (2026)
- [Taming CoT Obfuscation in VLMs: From Mechanistic Evidence to Activation Enforcement](https://mlsanity.com/paper/2609.24243/) (2026)
- [From Decorative to Load-Bearing: Task Difficulty Shapes the Causal Role of Chain-of-Thought](https://mlsanity.com/paper/2609.25366/) (2026)
- [Are Stated Reasoning Steps Causally Load-Bearing?](https://mlsanity.com/paper/2609.27038/) (2026)

## Abstract (arXiv)

Chain-of-thought (CoT) prompting boosts Large Language Models accuracy on multi-step tasks, yet whether the generated "thoughts" reflect the true internal reasoning process is unresolved. We present the first feature-level causal study of CoT faithfulness. Combining sparse autoencoders with activation patching, we extract monosemantic features from Pythia-70M and Pythia-2.8B while they tackle GSM8K math problems under CoT and plain (noCoT) prompting. Swapping a small set of CoT-reasoning features into a noCoT run raises answer log-probabilities significantly in the 2.8B model, but has no reliable effect in 70M, revealing a clear scale threshold. CoT also leads to significantly higher activation sparsity and feature interpretability scores in the larger model, signalling more modular internal computation. For example, the model's confidence in generating correct answers improves from 1.2 to 4.3. We introduce patch-curves and random-feature patching baselines, showing that useful CoT information is not only present in the top-K patches but widely distributed. Overall, our results indicate that CoT can induce more interpretable internal structures in high-capacity LLMs, validating its role as a structured prompting method.