mlsanityData to 7 Oct 2026

When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channel

arXiv 2605.11746 ↗First posted 12 May 2026cs.AIcited by 2

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-thought (CoT) traces are increasingly used both to improve language model capability and to audit model behavior, implicitly assuming that the visible trace remains synchronized with the computation that determines the answer. We test this assumption with a step-level Detect-Classify-Compare framework built around an answer-commitment proxy that is cross-validated with Patchscopes, tuned-lens probes, and causal direction ablation. Across nine models and seven reasoning benchmarks, latent commitment and explicit answer arrival align on only 61.9% of steps on average. The dominant mismatch pattern is confabulated continuation: 58.0% of detected mismatch events occur after the answer-commitment proxy has already stabilized while the trace continues producing deliberative-looking text, and a vacuousness analysis shows that the committed answer does not change during these steps. In architecture-matched Qwen2.5/DeepSeek-R1-Distill comparisons, the reasoning pipeline changes failure composition more than aggregate alignment, most clearly at 32B where confabulated steps decrease as contradictory states increase. Lower step-level alignment is also associated with larger CoT utility, suggesting that the settings that benefit most from CoT are often the least temporally faithful. Paired truncation and a complementary donor-corruption test further indicate that much post-commitment text is not load-bearing for the final answer. These findings suggest that CoT can remain useful while still being an unreliable report of when the answer was formed.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 63 of its 72 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

63 of 72 references.

Closest ancestors · 25

Same direction, newest first

  1. 2026Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Models27 Mar
  2. 2026Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?23 Mar
  3. 2026Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought5 Mar
  4. 2026Counterfactual Simulation Training for Chain-of-Thought Faithfulness24 Feb
  5. 2026Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning4 Feb
  6. 2025Reasoning Under Pressure: How do Training Incentives Influence Chain-of-Thought Monitorability?28 Nov
  7. 2025Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbosity31 Oct
  8. 2025Can Reasoning Models Obfuscate Reasoning? Stress-Testing Chain-of-Thought Monitorability21 Oct
  9. 2025FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning5 Oct
  10. 2025Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety15 Jul
  11. 2025When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors7 Jul
  12. 2025Reasoning Models Don't Always Say What They Think8 May
13 more
  1. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithful11 Mar
  2. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps20 Feb
  3. 2024Training Large Language Models to Reason in a Continuous Latent Space9 Dec
  4. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models15 Jun
  5. 2024Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning21 Feb
  6. 2024Chain-of-Thought Reasoning Without Prompting15 Feb
  7. 2023Preventing Language Models From Hiding Their Reasoning27 Oct
  8. 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul
  9. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting7 May
  10. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them17 Oct
  11. 2022Large Language Models are Zero-Shot Reasoners24 May
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
  13. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 38

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2024Qwen2.5 Technical Reportcited by 4,713
  7. 2023Let's Verify Step by Stepcited by 3,598
  8. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  11. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  12. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
26 more
  1. 2025Gemma 3 Technical Reportcited by 1,686
  2. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  4. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  5. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  6. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  7. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  8. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  9. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  10. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  11. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  12. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  13. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  14. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 523
  15. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 334
  16. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 225
  17. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  18. 2023Localizing Model Behavior with Path Patchingcited by 182
  19. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 158
  20. 2022Faithful Reasoning Using Large Language Modelscited by 130
  21. 2023Question Decomposition Improves the Faithfulness of Model-Generated Reasoningcited by 107
  22. 2023Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examplescited by 84
  23. 2023Benchmarking and Improving Generator-Validator Consistency of Language Modelscited by 43
  24. 2025LogitLens4LLMs: Extending Logit Lens Analysis to Modern Large Language Modelscited by 22
  25. 2025Performative Thinking? The Brittle Correlation Between CoT Length and Problem Complexitycited by 9
  26. 2026Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026When Chain-of-Thought Fails, the Solution Hides in the Hidden States
  2. 2026Reasoning that Travels: Dissecting How Chain-of-Thought Transfers Across Models
  3. 2026Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models
  4. 2026The Tell-Tale Trace: Detecting Reasoning Failures in LLMs Using Chain-of-Thought Dynamics
  5. 2026Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-Of-Thought Reasoning
  6. 2026The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages
  7. 2026The Coupling Tax: How Shared Token Budgets Undermine Visible Chain-of-Thought Under Fixed Output Limits
  8. 2026Probing the Trajectories of Reasoning Traces in Large Language Models