mlsanityData to 14 Oct 2026

FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoning

arXiv 2510.04040 ↗First posted 5 Oct 2025cs.AIcited by 26

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large language models (LLMs) increasingly rely on Chain-of-Thought (CoT) prompting to improve problem-solving and provide seemingly transparent explanations. However, growing evidence shows that CoT often fail to faithfully represent the underlying reasoning process, raising concerns about their reliability in high-risk applications. Although prior studies have focused on mechanism-level analyses showing that CoTs can be unfaithful, they leave open the practical challenge of deciding whether a specific trajectory is faithful to the internal reasoning of the model. To address this gap, we introduce FaithCoT-Bench, a unified benchmark for instance-level CoT unfaithfulness detection. Our framework establishes a rigorous task formulation that formulates unfaithfulness detection as a discriminative decision problem, and provides FINE-CoT (Faithfulness instance evaluation for Chain-of-Thought), an expert-annotated collection of over 1,000 trajectories generated by four representative LLMs across four domains, including more than 300 unfaithful instances with fine-grained causes and step-level evidence. We further conduct a systematic evaluation of eleven representative detection methods spanning counterfactual, logit-based, and LLM-as-judge paradigms, deriving empirical insights that clarify the strengths and weaknesses of existing approaches and reveal the increased challenges of detection in knowledge-intensive domains and with more advanced models. To the best of our knowledge, FaithCoT-Bench establishes the first comprehensive benchmark for instance-level CoT faithfulness, setting a solid basis for future research toward more interpretable and trustworthy reasoning in LLMs.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 49 of its 65 references that are in the map. First posted on arXiv 5 Oct 2025.

Builds on

49 of 65 references.

Closest ancestors · 19

Same direction, newest first

  1. 2025Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens2 Aug
  2. 2025Reasoning Models Don't Always Say What They Think8 May
  3. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models12 Mar
  4. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithful11 Mar
  5. 2025Measuring Chain of Thought Faithfulness by Unlearning Reasoning Steps20 Feb
  6. 2025Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models18 Feb
  7. 2024Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales?31 Oct
  8. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models15 Jun
  9. 2024Dissociation of Faithful and Unfaithful Reasoning in LLMs23 May
  10. 2024Chain of Thoughtlessness? An Analysis of CoT in Planning8 May
  11. 2024Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning21 Feb
  12. 2024How Interpretable are Reasoning Explanations from Prompting Large Language Models?19 Feb
7 more
  1. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Models18 Aug
  2. 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul
  3. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective24 May
  4. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting7 May
  5. 2023Faithful Chain-of-Thought Reasoning31 Jan
  6. 2022Large Language Models are Zero-Shot Reasoners24 May
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 30

  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  3. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  4. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
  5. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  6. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
  7. 2023Explainability for Large Language Models: A Surveycited by 333
  8. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
  9. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 289
  10. 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
  11. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  12. 2024Large Language Models for Data Annotation and Synthesis: A Surveycited by 205
18 more
  1. 2023Self-planning Code Generation with Large Language Modelscited by 124
  2. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  3. 2025In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agentscited by 106
  4. 2025AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learningcited by 87
  5. 2025Detecting Strategic Deception Using Linear Probescited by 68
  6. 2024Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Applicationcited by 56
  7. 2025Walk the Talk? Measuring the Faithfulness of Large Language Model Explanationscited by 45
  8. 2025Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systemscited by 43
  9. 2025On the Trustworthiness of Generative Foundation Models: Guideline, Assessment, and Perspectivecited by 41
  10. 2025BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attackscited by 38
  11. 2024The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Modelscited by 29
  12. 2025Measuring the Faithfulness of Thinking Drafts in Large Reasoning Modelscited by 23
  13. 2025Multi-Agent Simulator Drives Language Models for Legal Intensive Interactioncited by 22
  14. 2025How Well Can Reasoning Models Identify and Recover from Unhelpful Thoughts?cited by 11
  15. 2025Agentar-DeepFinance-100K: A Large-Scale Financial Dataset via Systematic Chain-of-Thought Synthesis Optimizationcited by 6
  16. 2025Evaluating Large Language Models for Cross-Lingual Retrievalcited by 5
  17. 2025LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inferencecited by 4
  18. 2025Towards Cognitively-Faithful Decision-Making Models to Improve AI Alignmentcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 26.

26 papers.

Sort:

By area:

  1. 2026Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?cited by 8
  2. 2026C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoningcited by 7
  3. 2026Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoningcited by 7
  4. 2026Reasoning Models Will Sometimes Lie About Their Reasoningcited by 6
  5. 2026Measuring Faithfulness Depends on How You Measure: Classifier Sensitivity in LLM Chain-of-Thought Evaluationcited by 4
  6. 2026Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Modelscited by 4
  7. 2026FACT-E: Causality-Inspired Evaluation for Trustworthy Chain-of-Thought Reasoningcited by 4
  8. 2026Faithfulness as Information Flow: Evaluating and Training Faithful Chain-of-Thought Reasoningcited by 3
  9. 2026Faithfulness Metrics Don't Measure Faithfulness: A Meta-Evaluation with Ground Truthcited by 3
  10. 2026Detecting Unfaithful Chain-of-Thought via Circuit-Guided Internal-External Discrepancycited by 3
  11. 2026SPD-Faith Bench: Diagnosing and Improving Faithfulness in Chain-of-Thought for Multimodal Large Language Modelscited by 2
  12. 2026The Reasoning Trap: An Information-Theoretic Bound on Closed-System Multi-Step LLM Reasoningcited by 2
14 more
  1. 2026When Reasoning Traces Become Performative: Step-Level Evidence that Chain-of-Thought Is an Imperfect Oversight Channelcited by 2
  2. 2026GRACE: Step-Level Benchmark for Faithful Reasoning over Contextcited by 2
  3. 2026When to Call an Apple Red: Humans Follow Introspective Rules, VLMs Don'tcited by 1
  4. 2026Decomposing and Steering Functional Metacognition in Large Language Modelscited by 1
  5. 2026Not All Explanations Simulate Equally: Comparing Verbalized Feature Attributions and Self-Generated Rationalescited by 1
  6. 2026Two Regimes of Chain-of-Thought Unfaithfulness: Metric-Based Detection Fails Where Models Are Wrongcited by 1
  7. 2026AISafetyBenchExplorer: A Metric-Aware Catalogue of AI Safety Benchmarks Reveals Fragmented Measurement and Weak Benchmark Governancecited by 0
  8. 2026The Cognitive Penalty: Ablating System 1 and System 2 Reasoning in Edge-Native SLMs for Decentralized Consensuscited by 0
  9. 2026What Does Chain-of-Thought Contribute at Probe Time? Evidence for Local Co-Occurrence Activationcited by 0
  10. 2026GeoFaith: A Spatio-Temporal Dual View of Faithful Chain-of-Thoughtcited by 0
  11. 2026Context-Masked Truncated Reasoning Audits for Answer-Key Dependence in LLM Tutorscited by 0
  12. 2026TsuGO: Probing Search Efficiency in LLM Reasoning via Go Life-and-Death Problemscited by 0
  13. 2026A Proposed Rubric for Evaluating Expressed Clinical Reasoning in Large Language Model Responsescited by 0
  14. 2026Do Time-Series QA Systems Read the Time Series? Evidence Use and Reasoning Reliabilitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning
  2. 2025A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Models
  3. 2026The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages
  4. 2025Is Chain-of-Thought Really Not Explainability? Chain-of-Thought Can Be Faithful without Hint Verbalization
  5. 2026MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models
  6. 2026The Tell-Tale Trace: Detecting Reasoning Failures in LLMs Using Chain-of-Thought Dynamics
  7. 2026Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations
  8. 2026Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models