Can We Trust AI Explanations? Evidence of Systematic Underreporting in Chain-of-Thought Reasoning
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
Abstract (arXiv)
When AI systems explain their reasoning step-by-step, practitioners often assume these explanations reveal what actually influenced the AI's answer. We tested this assumption by embedding hints into questions and measuring whether models mentioned them. In a study of over 9,000 test cases across 11 leading AI models, we found a troubling pattern: models almost never mention hints spontaneously, yet when asked directly, they admit noticing them. This suggests models see influential information but choose not to report it. Telling models they are being watched does not help. Forcing models to report hints works, but causes them to report hints even when none exist and reduces their accuracy. We also found that hints appealing to user preferences are especially dangerous-models follow them most often while reporting them least. These findings suggest that simply watching AI reasoning is not enough to catch hidden influences.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 11 of its 12 references that are in the map. First posted on arXiv 25 Dec 2025.
Builds on
11 of 12 references.
Closest ancestors · 6
Same direction, newest first
- 2025Reasoning Models Don't Always Say What They Think8 May
- 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting7 May
- 2022Large Language Models are Zero-Shot Reasoners24 May
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Models21 Mar
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan
Other references · 5
- 2020Measuring Massive Multitask Language Understandingcited by 7,768
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
- 2023Towards Understanding Sycophancy in Language Modelscited by 988
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Evaluating the False Trust Engendered by LLM Explanations
- 2026Reasoning Traces Shape Outputs but Models Won't Say So
- 2026Reasoning Models Will Sometimes Lie About Their Reasoning
- 2023In Search of Verifiability: Explanations Rarely Enable Complementary Performance in AI-Advised Decision Making
- 2026Selective Disclosure of Hidden Directives in Reasoning Models: Behavioral Asymmetry and Steering
- 2026On the Generalization of Steering Vectors for Chain-of-Thought Faithfulness
- 2026Do Models Know Why They Changed Their Mind? Interpretability and Faithfulness of Chain-of-Thought Under Knowledge Conflict
- 2025Thinking Out Loud: Do Reasoning Models Know When They're Right?