mlsanityData to 7 Oct 2026

Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thought

arXiv 2510.24941 ↗First posted 28 Oct 2025cs.LGcited by 14

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large language models can generate long chain-of-thought (CoT) reasoning, yet prior work suggests that CoT can be post-hoc rationalization rather than a faithful reflection of the computation through explicitly designed settings. In this work, we go further and propose a True Thinking Score (TTS) to quantify the causal contribution of each step in CoT to the model's final prediction in realistic reasoning problems. Across eleven models ranging from 1.5B to 1.1T parameters on common reasoning benchmarks, we find that CoTs often interleave true-thinking steps, which causally affect the final answer, with decorative-thinking steps, which appear useful but have little causal influence; Such decorative steps remain prevalent even for frontier models: Over 30% of steps in Kimi-K2.6 are decorative on MATH with TTS <= 0.005. Furthermore, TTS enables effective CoT pruning: removing 50% of CoT steps with the lowest TTS can largely maintain the performance. Self-training on these pruned CoTs reduces reasoning length by 66% while preserving performance on Nemotron3-Nano-30B. Finally, we provide a mechanistic analysis showing that LLMs can be steered in the latent space to engage or disengage with reasoning steps. Overall, our results reveal that frontier LLMs often verbalize reasoning steps that are not causally used, challenging both the efficiency and the trustworthiness of CoT.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 44 of its 61 references that are in the map. First posted on arXiv 28 Oct 2025.

Builds on

44 of 61 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety15 Jul
  2. 2025Thought Anchors: Which LLM Reasoning Steps Matter?23 Jun
  3. 2025Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning11 Jun
  4. 2025Reasoning Models Don't Always Say What They Think8 May
  5. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation14 Mar
  6. 2025Unlocking General Long Chain-of-Thought Reasoning Capabilities of Large Language Models via Representation Engineering14 Mar
  7. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithful11 Mar
  8. 2024Training Large Language Models to Reason in a Continuous Latent Space9 Dec
  9. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models15 Jun
  10. 2024Dissociation of Faithful and Unfaithful Reasoning in LLMs23 May
  11. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Models24 Apr
  12. 2024The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models11 Jan
2 more
  1. 2023Implicit Chain of Thought Reasoning via Knowledge Distillation2 Nov
  2. 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul

Other references · 30

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  6. 2025s1: Simple test-time scalingcited by 1,311
  7. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  8. 2024OpenAI o1 System Cardcited by 1,220
  9. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  10. 2023Steering Language Models With Activation Engineeringcited by 824
  11. 2024DeepSeek-V3 Technical Reportcited by 724
  12. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
18 more
  1. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  2. 2023Think before you speak: Training Language Models With Pause Tokenscited by 254
  3. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  4. 2025Deep Think with Confidencecited by 199
  5. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  6. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 175
  7. 2024Do Large Language Models Latently Perform Multi-Hop Reasoning?cited by 165
  8. 2023Linear Representations of Sentiment in Large Language Modelscited by 157
  9. 2025LLMs Encode Harmfulness and Refusal Separatelycited by 75
  10. 2025Are DeepSeek R1 And Other Reasoning Models More Faithful?cited by 69
  11. 2025Understanding Reasoning in Thinking Language Models via Steering Vectorscited by 69
  12. 2025SEAL: Steerable Reasoning Calibration of Large Language Models for Freecited by 65
  13. 2025ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Casescited by 51
  14. 2024A Language Model's Guide Through Latent Spacecited by 43
  15. 2025Language Models use Lookbacks to Track Beliefscited by 36
  16. 2025ThinkEdit: Interpretable Weight Editing to Mitigate Overly Short Thinking in Reasoning Modelscited by 17
  17. 2025Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoningcited by 16
  18. 2025On Reasoning Strength Planning in Large Reasoning Modelscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

  1. 2026Learning to Self-Verify Makes Language Models Better Reasonerscited by 18
  2. 2026Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvementcited by 8
  3. 2026The Hidden Signal of Verifier Strictness: Controlling and Improving Step-Wise Verification via Selective Latent Steeringcited by 5
  4. 2026ReasoningFlow: Discourse Structures for Understanding LLM Reasoning Tracescited by 5
  5. 2026Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Modelscited by 4
  6. 2026When Chain-of-Thought Fails, the Solution Hides in the Hidden Statescited by 3
  7. 2026Observable Patterns Are Not Explanations: A Causal-Geometric Analysis of Latent Reasoning Modelscited by 3
  8. 2026Directional Reasoning Trajectory Change (DRTC): Identifying Critical Trace Segments in Reasoning Modelscited by 1
  9. 2026Measuring and curing reasoning rigidity: from decorative chain-of-thought to genuine faithfulnesscited by 1
  10. 2026Reasoning Models Know What's Important, and Encode It in Their Activationscited by 1
  11. 2026Decoding the Critique Mechanism in Large Reasoning Modelscited by 0
  12. 2026PathCal: State-Aware Reflection-Marker Calibration for Efficient Reasoningcited by 0
2 more
  1. 2026It's the Problem, Not the Path: Budget and Difficulty Confounds in LLM Reasoning Trajectoriescited by 0
  2. 2026When Confidence Rises Too Early: Detecting Shortcut Reasoning via Premature Answer Commitmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?
  2. 2025What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT
  3. 2026Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations
  4. 2025A Formal Comparison Between Chain of Thought and Latent Thought
  5. 2024A Hopfieldian View-based Interpretation for Chain-of-Thought Reasoning
  6. 2026Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models
  7. 2025How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding
  8. 2026The Potential of CoT for Reasoning: A Closer Look at Trace Dynamics