mlsanityData to 14 Oct 2026

Question Decomposition Improves the Faithfulness of Model-Generated Reasoning

arXiv 2307.11768 ↗First posted 17 Jul 2023cs.CL cs.AI cs.LGcited by 108

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

As large language models (LLMs) perform more difficult tasks, it becomes harder to verify the correctness and safety of their behavior. One approach to help with this issue is to prompt LLMs to externalize their reasoning, e.g., by having them generate step-by-step reasoning as they answer a question (Chain-of-Thought; CoT). The reasoning may enable us to check the process that models use to perform tasks. However, this approach relies on the stated reasoning faithfully reflecting the model's actual reasoning, which is not always the case. To improve over the faithfulness of CoT reasoning, we have models generate reasoning by decomposing questions into subquestions. Decomposition-based methods achieve strong performance on question-answering tasks, sometimes approaching that of CoT while improving the faithfulness of the model's stated reasoning on several recently-proposed metrics. By forcing the model to answer simpler subquestions in separate contexts, we greatly increase the faithfulness of model-generated reasoning over CoT, while still achieving some of the performance gains of CoT. Our results show it is possible to improve the faithfulness of model-generated reasoning; continued improvements may lead to reasoning that enables us to verify the correctness and safety of LLM behavior.

Key facts. Cited by 108 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 40 of its 50 references that are in the map. First posted on arXiv 17 Jul 2023.

Builds on

40 of 50 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Iterated Decomposition: Improving Science Q&A by Supervising Reasoning Processes4 Jan
  2. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning15 Dec
  3. 2022Explanations from Large Language Models Make Small Reasoners Better13 Oct
  4. 2022ToKen: Task Decomposition and Knowledge Infusion for Few-Shot Hate Speech Detection25 May
  5. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning19 May
  6. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning6 May
  7. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Models30 Nov

Other references · 33

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  9. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  10. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  11. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  12. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
21 more
  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  2. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  3. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  4. 2022Galactica: A Large Language Model for Sciencecited by 744
  5. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  6. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  7. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
  8. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  9. 2023Faithful Chain-of-Thought Reasoningcited by 288
  10. 2020Measuring Association Between Labels and Free-Text Rationalescited by 160
  11. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  12. 2018Supervising strong learners by amplifying weak expertscited by 145
  13. 2022Learning by Distilling Contextcited by 138
  14. 2022Text and Patterns: For Effective Chain of Thought, It Takes Two to Tangocited by 128
  15. 2019Multi-hop Reading Comprehension through Question Decomposition and Rescoringcited by 119
  16. 2020Unsupervised Question Decomposition for Question Answeringcited by 118
  17. 2020Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?cited by 95
  18. 2022Successive Prompting for Decomposing Complex Questionscited by 94
  19. 2022Is a Question Decomposition Unit All We Need?cited by 37
  20. 2021Decomposing Complex Questions Makes Multi-Hop QA Easier and More Interpretablecited by 35
  21. 2022Complex Reading Comprehension Through Question Decompositioncited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 23; 2024: 34; 2025: 27; 2026: 24.

108 papers.

Sort:

By area:

  1. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  3. 2025Reasoning Models Don't Always Say What They Thinkcited by 415
  4. 2023Explainability for Large Language Models: A Surveycited by 333
  5. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 324
  6. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 266
  7. 2023Simple synthetic data reduces sycophancy in large language modelscited by 187
  8. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  9. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  10. 2025Chain-of-Thought Reasoning In The Wild Is Not Always Faithfulcited by 170
  11. 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 129
  12. 2023R-Tuning: Instructing Large Language Models to Say `I Don't Know'cited by 111
38 more
  1. 2024Prover-Verifier Games improve legibility of LLM outputscited by 85
  2. 2023On Measuring Faithfulness or Self-consistency of Natural Language Explanationscited by 73
  3. 2025Are DeepSeek R1 And Other Reasoning Models More Faithful?cited by 72
  4. 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialoguecited by 66
  5. 2023Preventing Language Models From Hiding Their Reasoningcited by 65
  6. 2024Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Samplingcited by 63
  7. 2024From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuningcited by 54
  8. 2024Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thoughtcited by 39
  9. 2025DeepRAG: Thinking to Retrieve Step by Step for Large Language Modelscited by 38
  10. 2023Re-Reading Improves Reasoning in Large Language Modelscited by 37
  11. 2023The ART of LLM Refinement: Ask, Refine, and Trustcited by 29
  12. 2024How Interpretable are Reasoning Explanations from Prompting Large Language Models?cited by 29
  13. 2024The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Modelscited by 29
  14. 2023Compositional preference models for aligning LMscited by 26
  15. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  16. 2024DeLLMa: Decision Making Under Uncertainty with Large Language Modelscited by 21
  17. 2024LLM-based Conversational AI Therapist for Daily Functioning Screening and Psychotherapeutic Intervention via Everyday Smart Devicescited by 21
  18. 2024Language Models in Dialogue: Conversational Maxims for Human-AI Interactionscited by 21
  19. 2025Teaching Models to Verbalize Reward Hacking in Chain-of-Thought Reasoningcited by 21
  20. 2025Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuningcited by 20
  21. 2024FaithLM: Towards Faithful Explanations for Large Language Modelscited by 19
  22. 2024Markov Chain of Thought for Efficient Mathematical Reasoningcited by 19
  23. 2024Breach By A Thousand Leaks: Unsafe Information Leakage in `Safe' AI Responsescited by 18
  24. 2024Model Tells Itself Where to Attend: Faithfulness Meets Automatic Attention Steeringcited by 17
  25. 2024DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning Trajectories Searchcited by 17
  26. 2026Counterfactual Simulation Training for Chain-of-Thought Faithfulnesscited by 17
  27. 2023SCREWS: A Modular Framework for Reasoning with Revisionscited by 16
  28. 2024Automatic Curriculum Expert Iteration for Reliable LLM Reasoningcited by 16
  29. 2024Have the VLMs Lost Confidence? A Study of Sycophancy in VLMscited by 15
  30. 2025Don't Let It Hallucinate: Premise Verification via Retrieval-Augmented Logical Reasoningcited by 15
  31. 2024GermanPartiesQA: Benchmarking Commercial Large Language Models and AI Companions for Political Alignment and Sycophancycited by 14
  32. 2025Vendi-RAG: Adaptively Trading-Off Diversity And Quality Significantly Improves Retrieval Augmented Generation With LLMscited by 14
  33. 2025Knowledge Graph-extended Retrieval Augmented Generation for Question Answeringcited by 14
  34. 2025Conformal Language Model Reasoning with Coherent Factualitycited by 14
  35. 2024Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoningcited by 13
  36. 2024CliqueParcel: An Approach For Batching LLM Prompts That Jointly Optimizes Efficiency And Faithfulnesscited by 12
  37. 2025A Closer Look at Bias and Chain-of-Thought Faithfulness of Large (Vision) Language Modelscited by 12
  38. 2023AutoHall: Automated Factuality Hallucination Dataset Generation for Large Language Modelscited by 10

50 most cited of 108 All 108 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Faithful Reasoning Using Large Language Models
  2. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought
  3. 2023Measuring Faithfulness in Chain-of-Thought Reasoning
  4. 2024textttLM^texttt2: A Simple Society of Language Models Solves Complex Reasoning
  5. 2026Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment
  6. 2024FLARE: Faithful Logic-Aided Reasoning and Exploration
  7. 2025Early Stopping Chain-of-thoughts in Large Language Models
  8. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models