mlsanityData to 7 Oct 2026

FaiRR: Faithful and Robust Deductive Reasoning over Natural Language

arXiv 2203.10261 ↗First posted 19 Mar 2022cs.CL cs.AI cs.LGcited by 29

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Transformers have been shown to be able to perform deductive reasoning on a logical rulebase containing rules and statements written in natural language. Recent works show that such models can also produce the reasoning steps (i.e., the proof graph) that emulate the model's logical reasoning process. Currently, these black-box models generate both the proof graph and intermediate inferences within the same model and thus may be unfaithful. In this work, we frame the deductive logical reasoning task by defining three modular components: rule selection, fact selection, and knowledge composition. The rule and fact selection steps select the candidate rule and facts to be used and then the knowledge composition combines them to generate new inferences. This ensures model faithfulness by assured causal relation from the proof step to the inference reasoning. To test our framework, we propose FaiRR (Faithful and Robust Reasoner) where the above three components are independently modeled by transformers. We observe that FaiRR is robust to novel language perturbations, and is faster at inference than previous works on existing reasoning datasets. Additionally, in contrast to black-box generative models, the errors made by FaiRR are more interpretable due to the modular approach.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 28 of its 38 references that are in the map. First posted on arXiv 19 Mar 2022.

Builds on

28 of 38 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  6. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  7. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  8. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  9. 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 309
  10. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
  11. 2020Transformers as Soft Reasoners over Languagecited by 265
  12. 2019Causality for Machine Learningcited by 242
16 more
  1. 2020Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Biascited by 183
  2. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 177
  3. 2017End-to-End Differentiable Provingcited by 159
  4. 2019CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Textcited by 155
  5. 2018Invariance, Causality and Robustnesscited by 123
  6. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
  7. 2019QuaRTz: An Open-Domain Dataset of Qualitative Relationship Questionscited by 89
  8. 2019Reasoning Over Paragraph Effects in Situationscited by 79
  9. 2021Counterfactual Invariance to Spurious Correlations: Why and How to Pass Stress Testscited by 77
  10. 2018Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMscited by 67
  11. 2018A Survey on Semantic Parsingcited by 62
  12. 2020PRover: Proof Generation for Interpretable Reasoning over Rulescited by 53
  13. 2018Causal Reasoning for Algorithmic Fairnesscited by 51
  14. 2021Discretized Integrated Gradients for Explaining Language Modelscited by 45
  15. 2019NLProlog: Reasoning with Weak Unification for Question Answering in Natural Languagecited by 36
  16. 2021multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoningcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 11; 2024: 10; 2025: 1; 2026: 2.

29 papers.

Sort:

By area:

  1. 2023Natural Language Reasoning, A Surveycited by 61
  2. 2024A Survey on Deep Learning for Theorem Provingcited by 55
  3. 2022Generating Natural Language Proofs with Verifier-Guided Searchcited by 52
  4. 2023Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Modelscited by 37
  5. 2024Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpuscited by 29
  6. 2024How Interpretable are Reasoning Explanations from Prompting Large Language Models?cited by 27
  7. 2022RobustLR: Evaluating Robustness to Logical Perturbation in Deductive Reasoningcited by 22
  8. 2023Learning Deductive Reasoning from Synthetic Corpus based on Formal Logiccited by 18
  9. 2023RECKONING: Reasoning through Dynamic Knowledge Encodingcited by 17
  10. 2024Quantifying the Capabilities of LLMs across Scale and Precisioncited by 17
  11. 2024Symbolic Working Memory Enhances Language Models for Complex Rule Applicationcited by 17
  12. 2023Faithful Question Answering with Monte-Carlo Planningcited by 16
17 more
  1. 2024Enhancing Ethical Explanations of Large Language Models through Iterative Symbolic Refinementcited by 16
  2. 2023Iterated Decomposition: Improving Science Q&A by Supervising Reasoning Processescited by 15
  3. 2023GLoRE: Evaluating Logical Reasoning of Large Language Modelscited by 13
  4. 2024Reverse Question Answering: Can an LLM Write a Question so Hard (or Bad) that it Can't Answer?cited by 13
  5. 2022RLET: A Reinforcement Learning Based Approach for Explainable QA with Entailment Treescited by 7
  6. 2023Empower Nested Boolean Logic via Self-Supervised Curriculum Learningcited by 7
  7. 2023CLOMO: Counterfactual Logical Modification with Large Language Modelscited by 7
  8. 2022Can Pretrained Language Models (Yet) Reason Deductively?cited by 5
  9. 2022APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoningcited by 5
  10. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 5
  11. 2024Improving Arithmetic Reasoning Ability of Large Language Models through Relation Tuples, Verification and Dynamic Feedbackcited by 4
  12. 2024SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learningcited by 3
  13. 2024RISCORE: Enhancing In-Context Riddle Solving in Language Models through Context-Reconstructed Example Augmentationcited by 3
  14. 2026NL2LOGIC: AST-Guided Translation of Natural Language into First-Order Logic with Large Language Modelscited by 3
  15. 2023Empirical Investigation of Neural Symbolic Reasoning Strategiescited by 1
  16. 2023Explainable Verbal Reasoner Plus (EVR+): A Natural Language Reasoning Framework that Supports Diverse Compositional Reasoningcited by 1
  17. 2025Automated Annotation of Evolving Corpora for Augmenting Longitudinal Network Data: A Framework Integrating Large Language Models and Expert Knowledgecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Toward Honest Language Models for Deductive Reasoning
  2. 2025Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation
  3. 2023Examining the Emergence of Deductive Reasoning in Generative Language Models
  4. 2025Can Large Language Models Learn Formal Logic? A Data-Driven Training and Evaluation Framework
  5. 2025From Hypothesis to Premises: LLM-based Backward Logical Reasoning with Selective Symbolic Translation
  6. 2025Investigating the Robustness of Deductive Reasoning with Large Language Models
  7. 2025JustLogic: A Comprehensive Benchmark for Evaluating Deductive Reasoning in Large Language Models
  8. 2025Patterns Over Principles: The Fragility of Inductive Reasoning in LLMs under Noisy Observations