mlsanityData to 14 Oct 2026

Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning

arXiv 2210.12217 ↗First posted 21 Oct 2022cs.AI cs.CLcited by 38

Not on the map. Why?

Cited by 20 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Our goal is a question-answering (QA) system that can show how its answers are implied by its own internal beliefs via a systematic chain of reasoning. Such a capability would allow better understanding of why a model produced the answer it did. Our approach is to recursively combine a trained backward-chaining model, capable of generating a set of premises entailing an answer hypothesis, with a verifier that checks that the model itself believes those premises (and the entailment itself) through self-querying. To our knowledge, this is the first system to generate multistep chains that are both faithful (the answer follows from the reasoning) and truthful (the chain reflects the system's own internal beliefs). In evaluation using two different datasets, users judge that a majority (70%+) of generated chains clearly show how an answer follows from a set of facts - substantially better than a high-performance baseline - while preserving answer accuracy. By materializing model beliefs that systematically support an answer, new opportunities arise for understanding the model's system of belief, and diagnosing and correcting its misunderstandings when an answer is wrong.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). Builds on 31 of its 43 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

31 of 43 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  5. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  6. 2019Language Models as Knowledge Bases?cited by 1,936
  7. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  8. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  9. 2021Fast Model Editing at Scalecited by 498
  10. 2019COMET: Commonsense Transformers for Automatic Knowledge Graph Constructioncited by 450
  11. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
  12. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 334
19 more
  1. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 324
  2. 2020Transformers as Soft Reasoners over Languagecited by 270
  3. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 243
  4. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
  5. 2021Explaining Answers with Entailment Treescited by 152
  6. 2022Rationale-Augmented Ensembles in Language Modelscited by 111
  7. 2018Transforming Question Answering Datasets Into Natural Language Inference Datasetscited by 99
  8. 2019QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questionscited by 89
  9. 2019A Logic-Driven Framework for Consistency of Neural Modelscited by 64
  10. 2021BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Beliefcited by 57
  11. 2022Generating Natural Language Proofs with Verifier-Guided Searchcited by 52
  12. 2021General-Purpose Question-Answering with Macawcited by 40
  13. 2022Towards Teachable Reasoning Systems: Using a Dynamic Memory of User Feedback for Continual System Improvementcited by 37
  14. 2022Natural Language Deduction through Search over Statement Compositionscited by 35
  15. 2021Flexible Generation of Natural Language Deductionscited by 28
  16. 2022Entailment Tree Explanations via Iterative Retrieval-Generation Reasonercited by 26
  17. 2020Paragraph-level Commonsense Transformers with Recurrent Memorycited by 23
  18. 2022METGEN: A Module-Based Entailment Tree Generation Framework for Answer Explanationcited by 17
  19. 2020Braid: Weaving Symbolic and Neural Knowledge into Coherent Logical Explanationscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 17; 2024: 12; 2025: 7; 2026: 2.

38 papers.

Sort:

By area:

  1. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  2. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 391
  3. 2023Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoningcited by 328
  4. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  5. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  6. 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Proverscited by 168
  7. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thoughtcited by 87
  8. 2023Natural Language Reasoning, A Surveycited by 61
  9. 2024A Survey on Deep Learning for Theorem Provingcited by 57
  10. 2023DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agentscited by 52
  11. 2025Empowering LLMs with Logical Reasoning: A Comprehensive Surveycited by 51
  12. 2023Vera: A General-Purpose Plausibility Estimation Model for Commonsense Statementscited by 44
26 more
  1. 2024Argumentative Large Language Models for Explainable and Contestable Claim Verificationcited by 36
  2. 2023Logical Reasoning over Natural Language as Knowledge Representation: A Surveycited by 32
  3. 2023Say What You Mean! Large Language Models Speak Too Positively about Negative Commonsense Knowledgecited by 32
  4. 2023Language Models can be Logical Solverscited by 24
  5. 2024Deductive Closure Training of Language Models for Coherence, Accuracy, and Updatabilitycited by 22
  6. 2024A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagationcited by 21
  7. 2023RECKONING: Reasoning through Dynamic Knowledge Encodingcited by 18
  8. 2024BIRD: A Trustworthy Bayesian Inference Framework for Large Language Modelscited by 18
  9. 2023Faithful Question Answering with Monte-Carlo Planningcited by 16
  10. 2023Language Models with Rationalitycited by 15
  11. 2023Self-Contradictory Reasoning Evaluation and Detectioncited by 14
  12. 2024Are Machines Better at Complex Reasoning? Unveiling Human-Machine Inference Gaps in Entailment Verificationcited by 13
  13. 2024Enhancing Systematic Decompositional Natural Language Inference Using Informal Logiccited by 12
  14. 2025Commonsense Video Question Answering through Video-Grounded Entailment Tree Reasoningcited by 12
  15. 2025VeriCoT: Neuro-symbolic Chain-of-Thought Validation via Logical Consistency Checkscited by 11
  16. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  17. 2025Instantiation-based Formalization of Logical Reasoning Tasks using Language Models and Logical Solverscited by 7
  18. 2025Mitigating Spurious Correlations in LLMs via Causality-Aware Post-Trainingcited by 5
  19. 2024Thought-Like-Pro: Enhancing Reasoning of Large Language Models through Self-Driven Prolog-based Chain-of-Thoughtcited by 4
  20. 2024SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learningcited by 3
  21. 2025RECAP: Transparent Inference-Time Emotion Alignment for Medical Dialogue Systemscited by 3
  22. 2023C5: Towards Better Conversation Comprehension and Contextual Continuity for ChatGPTcited by 1
  23. 2024From Models to Microtheories: Distilling a Model's Topical Knowledge for Grounded Question Answeringcited by 1
  24. 2026ANCHOR: Abductive Network Construction with Hierarchical Orchestration for Reliable Probability Inference in Large Language Modelscited by 1
  25. 2024Counterfactual Samples Constructing and Training for Commonsense Statements Estimationcited by 0
  26. 2026The Maskability Index: Predicting Task-Objective Alignment in Pretrained Language Modelscited by 0