Do Models Know Why They Changed Their Mind? Interpretability and Faithfulness of Chain-of-Thought Under Knowledge Conflict
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
Abstract (arXiv)
When a language model sees a document contradicting its training knowledge, it must choose: follow the document or trust itself. Prior work proved this choice depends on how well-known the fact is. We ask: does the model's chain-of-thought (CoT) reasoning faithfully report this mechanism? We introduce introspective faithfulness and test it across 200 questions, 8 models, and 4 prompt conditions. We find CoT reasoning is highly stable across opposite decisions: flip pairs retain 96% of same-answer similarity (d=0.34; confirmed by ROUGE-L, d=0.45). Yet self-rated confidence carries a faint genuine signal: for obscure facts where entity fame is uninformative, confidence still predicts decisions (p<0.001) and tracks item-level knowledge (r=0.134). GPT-4o is the only model with statistically reliable reasoning-decision coupling. Claude Sonnet 4.6 shows the widest confidence range (SD=1.39) but near-zero pooled correlation because the confidence-decision relationship reverses between conditions; a temperature ablation confirms this is model-specific. Internal thinking tokens show greater decision-sensitivity than user-facing CoT (p=0.033). CoT decomposes into a decision-invariant knowledge display (~96%) and a thin confidence layer with weak but real signal. For monitoring: read confidence, not the argument.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 0 of its 22 references that are in the map. First posted on arXiv 26 May 2026.
Builds on
0 of 22 references.
No references here yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?
- 2023Measuring Faithfulness in Chain-of-Thought Reasoning
- 2024Chain-of-Thought Unfaithfulness as Disguised Accuracy
- 2026From Concept Alignment to Causal Grounding: An Intervention Test of Chain-of-Thought Faithfulness
- 2026Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning
- 2026Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoning
- 2025FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning
- 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps