mlsanityData to 7 Oct 2026

Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning

arXiv 2402.13950 ↗First posted 21 Feb 2024cs.CLcited by 108

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Large language models (LLMs) have been shown to perform better when asked to reason step-by-step before answering a question. However, it is unclear to what degree the model's final answer is faithful to the stated reasoning steps. In this paper, we perform a causal mediation analysis on twelve LLMs to examine how intermediate reasoning steps generated by the LLM influence the final outcome and find that LLMs do not reliably use their intermediate reasoning steps when generating an answer. To address this issue, we introduce FRODO, a framework to tailor small-sized LMs to generate correct reasoning steps and robustly reason over these steps. FRODO consists of an inference module that learns to generate correct reasoning steps using an implicit causal reward function and a reasoning module that learns to faithfully reason over these intermediate inferences using a counterfactual and causal preference objective. Our experiments show that FRODO significantly outperforms four competitive baselines. Furthermore, FRODO improves the robustness and generalization ability of the reasoning LM, yielding higher performance on out-of-distribution test sets. Finally, we find that FRODO's rationales are more faithful to its final answer predictions than standard supervised fine-tuning.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 40 of its 61 references that are in the map. First posted on arXiv 21 Feb 2024.

Builds on

40 of 61 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023Measuring Faithfulness in Chain-of-Thought Reasoning17 Jul
  2. 2023Symbolic Chain-of-Thought Distillation: Small Models Can Also "Think" Step-by-Step24 Jun
  3. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting7 May
  4. 2023SCOTT: Self-Consistent Chain-of-Thought Distillation3 May
  5. 2023Faithful Chain-of-Thought Reasoning31 Jan
  6. 2022Distilling Reasoning Capabilities into Smaller Language Models1 Dec
  7. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them17 Oct
  8. 2022Large Language Models are Zero-Shot Reasoners24 May

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  7. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  8. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  9. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 736
  10. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  11. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  12. 2023Language Models can Solve Computer Taskscited by 371
20 more
  1. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  2. 2023How Language Model Hallucinations Can Snowballcited by 310
  3. 2013Direct and Indirect Effectscited by 276
  4. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 249
  5. 2019QASC: A Dataset for Question Answering via Sentence Compositioncited by 248
  6. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  7. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 203
  8. 2020Measuring Association Between Labels and Free-Text Rationalescited by 159
  9. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 127
  10. 2022Explanations from Large Language Models Make Small Reasoners Bettercited by 123
  11. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  12. 2023RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputscited by 104
  13. 2020Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?cited by 95
  14. 2023FLIRT: Feedback Loop In-context Red Teamingcited by 81
  15. 2018QuaRel: A Dataset and Models for Answering Questions about Qualitative Relationshipscited by 46
  16. 2022Rainier: Reinforced Knowledge Introspector for Commonsense Question Answeringcited by 43
  17. 2023Chain-of-Thought in Neural Code Generation: From and For Lightweight Language Modelscited by 27
  18. 2023Tailoring Self-Rationalizers with Multi-Reward Distillationcited by 21
  19. 2023Crystal: Introspective Reasoners Reinforced with Self-Feedbackcited by 15
  20. 2021COINS: Dynamically Generating COntextualized Inference Rules for Narrative Story Completioncited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 51; 2026: 47.

108 papers.

Sort:

By area:

  1. 2025Reasoning Models Don't Always Say What They Thinkcited by 388
  2. 2024Multi-Step Reasoning with Large Language Models, a Surveycited by 140
  3. 2025When Chain of Thought is Necessary, Language Models Struggle to Evade Monitorscited by 81
  4. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Modelscited by 70
  5. 2025Are DeepSeek R1 And Other Reasoning Models More Faithful?cited by 69
  6. 2025Evaluating Step-by-step Reasoning Traces: A Surveycited by 62
  7. 2025CoT Red-Handed: Stress Testing Chain-of-Thought Monitoringcited by 49
  8. 2025Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!cited by 36
  9. 2024How Likely Do LLMs with CoT Mimic Human Reasoning?cited by 34
  10. 2026This human study did not involve human subjects: Validating LLM simulations as behavioral evidencecited by 26
  11. 2025FaithCoT-Bench: Benchmarking Instance-Level Faithfulness of Chain-of-Thought Reasoningcited by 25
  12. 2024Towards Better Chain-of-Thought: A Reflection on Effectiveness and Faithfulnesscited by 20
38 more
  1. 2024Rational Metareasoning for Large Language Modelscited by 20
  2. 2025Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuningcited by 19
  3. 2025How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencodingcited by 19
  4. 2025On the Faithfulness of Visual Thinking: Measurement and Enhancementcited by 17
  5. 2025Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Modelscited by 16
  6. 2025When Thinking Drifts: Evidential Grounding for Robust Video Reasoningcited by 15
  7. 2025Measuring Chain-of-Thought Monitorability Through Faithfulness and Verbositycited by 15
  8. 2026Counterfactual Simulation Training for Chain-of-Thought Faithfulnesscited by 15
  9. 2024Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Modelscited by 14
  10. 2025MetaOpenFOAM 2.0: Large Language Model Driven Chain of Thought for Automating CFD Simulation and Post-Processingcited by 13
  11. 2025A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Modelscited by 12
  12. 2025Do Cognitively Interpretable Reasoning Traces Improve LLM Performance?cited by 11
  13. 2025Truthful or Fabricated? Using Causal Attribution to Mitigate Reward Hacking in Explanationscited by 9
  14. 2025FRIT: Using Causal Importance to Improve Chain-of-Thought Faithfulnesscited by 9
  15. 2025Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbationscited by 9
  16. 2026Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoningcited by 9
  17. 2026RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Modelscited by 9
  18. 2024Chain-of-Probe: Examining the Necessity and Accuracy of CoT Step-by-Stepcited by 8
  19. 2025Do LLM Agents Know How to Ground, Recover, and Assess? A Benchmark for Epistemic Competence in Information-Seeking Agentscited by 7
  20. 2026Analyzing and Improving Chain-of-Thought Monitorability Through Information Theorycited by 7
  21. 2026Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoningcited by 7
  22. 2025Unraveling Misinformation Propagation in LLM Reasoningcited by 6
  23. 2025Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generationcited by 6
  24. 2026Simulated Ignorance Fails: A Systematic Study of LLM Behaviors on Forecasting Problems Before Model Knowledge Cutoffcited by 6
  25. 2025From Five Dimensions to Many: Large Language Models as Precise and Interpretable Psychological Profilerscited by 5
  26. 2025State over Tokens: Characterizing the Role of Reasoning Tokenscited by 5
  27. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 5
  28. 2024Markovian Transformers for Informative Language Modelingcited by 4
  29. 2025OwkinZero: Accelerating Biological Discovery with AIcited by 4
  30. 2025From Faithfulness to Correctness: Generative Reward Models that Think Criticallycited by 4
  31. 2025Unspoken Hints: Accuracy Without Acknowledgement in LLM Reasoningcited by 4
  32. 2025Putting on the Thinking Hats: A Survey on Chain of Thought Fine-tuning from the Perspective of Human Reasoning Mechanismcited by 4
  33. 2026Reasoning Models Will Sometimes Lie About Their Reasoningcited by 4
  34. 2026Are LLMs Reliable Code Reviewers? Systematic Overcorrection in Requirement Conformance Judgementcited by 4
  35. 2026Detecting and Suppressing Reward Hacking with Gradient Fingerprintscited by 4
  36. 2026ReasonOps: Operator Segmentation for LLM Reasoning Tracescited by 4
  37. 2024Topic Classification of Case Law Using a Large Language Model and a New Taxonomy for UK Law: AI Insights into Summary Judgmentcited by 3
  38. 2024Graph-Guided Textual Explanation Generation Frameworkcited by 3

50 most cited of 108 All 108 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Faithful Reasoning Using Large Language Models
  2. 2026C2-Faith: Benchmarking LLM Judges for Causal and Coverage Faithfulness in Chain-of-Thought Reasoning
  3. 2025Beyond the Last Answer: Your Reasoning Trace Uncovers More than You Think
  4. 2025What Defines Good Reasoning in LLMs? Dissecting Reasoning Steps with Multi-Aspect Evaluation
  5. 2024CSCE: Boosting LLM Reasoning by Simultaneous Enhancing of Causal Significance and Consistency
  6. 2025Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning
  7. 2026Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models
  8. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought