mlsanityData to 14 Oct 2026

A Theoretical Understanding of Chain-of-Thought: Coherent Reasoning and Error-Aware Demonstration

arXiv 2410.16540 ↗First posted 21 Oct 2024cs.CL cs.AI cs.LGcited by 13

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Few-shot Chain-of-Thought (CoT) prompting has demonstrated strong performance in improving the reasoning capabilities of large language models (LLMs). While theoretical investigations have been conducted to understand CoT, the underlying transformer used in these studies isolates the CoT reasoning process into separated in-context learning steps (Stepwise ICL). In this work, we theoretically show that, compared to Stepwise ICL, the transformer gains better error correction ability and more accurate predictions if the reasoning from earlier steps (Coherent CoT) is integrated. Given that this coherent reasoning changes the behavior of the transformer, we further investigate the sensitivity of the transformer with Coherent CoT when the demonstration examples are corrupted at the inference stage. Our theoretical results indicate that the transformer is more sensitive to errors in intermediate reasoning steps than the final outcome. Building upon this observation, we propose an improvement on CoT by incorporating both correct and incorrect reasoning paths in the demonstration. Our experiments validate the effectiveness of the proposed approach.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 32 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

28 of 32 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problems20 Feb
  2. 2024The Impact of Reasoning Step Length on Large Language Models10 Jan
  3. 2023Why Can Large Language Models Generate Correct Chain-of-Thoughts?20 Oct
  4. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Models18 Aug
  5. 2023Analyzing Chain-of-Thought Prompting in Large Language Models via Gradient-based Feature Attributions25 Jul
  6. 2023Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learning30 May
  7. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective24 May
  8. 2023Faithful Chain-of-Thought Reasoning31 Jan
  9. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters20 Dec
  10. 2022Automatic Chain of Thought Prompting in Large Language Models7 Oct
  11. 2022Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango16 Sep
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar

Other references · 16

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  4. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  5. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 872
  6. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
  7. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  8. 2023Trained Transformers Learn Linear Models In-Contextcited by 337
  9. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 298
  10. 2023Transformers learn to implement preconditioned gradient descent for in-context learningcited by 279
  11. 2023In-Context Convergence of Transformerscited by 115
  12. 2023Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performancecited by 102
4 more
  1. 2024Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimalitycited by 74
  2. 2023Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Contextcited by 63
  3. 2024Superiority of Multi-Head Attention in In-Context Linear Regressioncited by 17
  4. 2024Theoretical Understanding of In-Context Learning in Shallow Transformers with Unstructured Datacited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 9; 2026: 2.

13 papers.

Sort:

By area:

  1. 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 183
  2. 2025Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Modelscited by 95
  3. 2024Beyond Examples: High-level Automated Reasoning Paradigm in In-Context Learning via MCTScited by 54
  4. 2025Knowing Before Saying: LLM Representations Encode Information About Chain-of-Thought Success Before Completioncited by 23
  5. 2025Reasoning Beyond Limits: Advances and Open Problems for LLMscited by 16
  6. 2025GateLens: A Reasoning-Enhanced LLM Agent for Automotive Software Release Analyticscited by 4
  7. 2025Reasoning Can Hurt the Inductive Abilities of Large Language Modelscited by 4
  8. 2026FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoningcited by 4
  9. 2025Bounds of Chain-of-Thought Robustness: Reasoning Steps, Embed Norms, and Beyondcited by 2
  10. 2025Team Anotheroption at SemEval-2025 Task 8: Bridging the Gap Between Open-Source and Proprietary LLMs in Table QAcited by 1
  11. 2024A Multimodal Social Agentcited by 0
  12. 2025Conversational Orientation Reasoning: Egocentric-to-Allocentric Navigation with Multimodal Chain-of-Thoughtcited by 0
1 more
  1. 2026Demonstrations, CoT, and Prompting: A Theoretical Analysis of ICLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Chain-of-Thought Reasoning Without Prompting
  2. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  3. 2025Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
  4. 2024A Hopfieldian View-based Interpretation for Chain-of-Thought Reasoning
  5. 2025To Think or Not to Think: The Hidden Cost of Meta-Training with Excessive CoT Examples
  6. 2025The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning
  7. 2024Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysis
  8. 2022Large Language Models are Zero-Shot Reasoners