# A Hopfieldian View-based Interpretation for Chain-of-Thought Reasoning

arXiv: https://arxiv.org/abs/2406.12255 · HTML: https://mlsanity.com/paper/2406.12255/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 34 of its 86 references that are in the map.
- First posted on arXiv 18 Jun 2024.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)

## Builds on (most cited first, 30 of 34)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Chain-of-Thought Reasoning Without Prompting](https://mlsanity.com/paper/2402.10200/) (2024)
- [ReFT: Representation Finetuning for Language Models](https://mlsanity.com/paper/2404.03592/) (2024)
- [Let's Think Dot by Dot: Hidden Computation in Transformer Language Models](https://mlsanity.com/paper/2404.15758/) (2024)
- [Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis](https://mlsanity.com/paper/2304.04675/) (2023)
- [The Impact of Reasoning Step Length on Large Language Models](https://mlsanity.com/paper/2401.04925/) (2024)
- [Rethinking with Retrieval: Faithful Large Language Model Inference](https://mlsanity.com/paper/2301.00303/) (2022)
- [Faithful Reasoning Using Large Language Models](https://mlsanity.com/paper/2208.14271/) (2022)
- [Chain of Thoughtlessness? An Analysis of CoT in Planning](https://mlsanity.com/paper/2405.04776/) (2024)
- [Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners](https://mlsanity.com/paper/2305.14825/) (2023)
- [MoRAL: MoE Augmented LoRA for LLMs' Lifelong Learning](https://mlsanity.com/paper/2402.11260/) (2024)
- [Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Code](https://mlsanity.com/paper/2311.07989/) (2023)
- [How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning](https://mlsanity.com/paper/2402.18312/) (2024)
- [The Expressive Power of Transformers with Chain of Thought](https://mlsanity.com/paper/2310.07923/) (2023)
- [Multi-hop Question Answering under Temporal Knowledge Editing](https://mlsanity.com/paper/2404.00492/) (2024)
- [Editable Concept Bottleneck Models](https://mlsanity.com/paper/2405.15476/) (2024)
- [Differentially Private Natural Language Models: Recent Advances and Future Directions](https://mlsanity.com/paper/2301.09112/) (2023)
- [Improving Interpretation Faithfulness for Vision Transformers](https://mlsanity.com/paper/2311.17983/) (2023)

## Built on by (most cited first, 15 of 15)

- [When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models](https://mlsanity.com/paper/2508.02087/) (2025)
- [Semi-supervised Concept Bottleneck Models](https://mlsanity.com/paper/2406.18992/) (2024)
- [Adaptive Multi-Subspace Representation Steering for Attribute Alignment in Large Language Models](https://mlsanity.com/paper/2508.10599/) (2025)
- [Mechanistic Unveiling of Transformer Circuits: Self-Influence as a Key to Model Reasoning](https://mlsanity.com/paper/2502.09022/) (2025)
- [MQA-KEAL: Multi-hop Question Answering under Knowledge Editing for Arabic Language](https://mlsanity.com/paper/2409.12257/) (2024)
- [Towards Multi-dimensional Explanation Alignment for Medical Classification](https://mlsanity.com/paper/2410.21494/) (2024)
- [COMPKE: Complex Question Answering under Knowledge Editing](https://mlsanity.com/paper/2506.00829/) (2025)
- [Controlling Repetition in Protein Language Models](https://mlsanity.com/paper/2602.00782/) (2026)
- [Beyond Scalars: Evaluating and Understanding LLM Reasoning via Geometric Progress and Stability](https://mlsanity.com/paper/2603.10384/) (2026)
- [Towards Reasoning-Preserving Unlearning in Multimodal Large Language Models](https://mlsanity.com/paper/2512.17911/) (2025)
- [Faithful Interpretation for Graph Neural Networks](https://mlsanity.com/paper/2410.06950/) (2024)
- [Stable Vision Concept Transformers for Medical Diagnosis](https://mlsanity.com/paper/2506.05286/) (2025)
- [Understanding the Dynamics of Demonstration Conflict in In-Context Learning](https://mlsanity.com/paper/2603.04464/) (2026)
- [OncoReason: Structuring Clinical Reasoning in LLMs for Robust and Interpretable Survival Prediction](https://mlsanity.com/paper/2510.17532/) (2025)
- [Multi-Adapter Representation Interventions via Energy Calibration](https://mlsanity.com/paper/2605.28722/) (2026)

## Abstract (arXiv)

Chain-of-Thought (CoT) holds a significant place in augmenting the reasoning performance for large language models (LLMs). While some studies focus on improving CoT accuracy through methods like retrieval enhancement, yet a rigorous explanation for why CoT achieves such success remains unclear. In this paper, we analyze CoT methods under two different settings by asking the following questions: (1) For zero-shot CoT, why does prompting the model with "let's think step by step" significantly impact its outputs? (2) For few-shot CoT, why does providing examples before questioning the model could substantially improve its reasoning ability? To answer these questions, we conduct a top-down explainable analysis from the Hopfieldian view and propose a Read-and-Control approach for controlling the accuracy of CoT. Through extensive experiments on seven datasets for three different tasks, we demonstrate that our framework can decipher the inner workings of CoT, provide reasoning error localization, and control to come up with the correct reasoning path.