# Multi-Step Deductive Reasoning Over Natural Language: An Empirical Study on Out-of-Distribution Generalisation arXiv: https://arxiv.org/abs/2207.14000 ยท HTML: https://mlsanity.com/paper/2207.14000/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. - Builds on 21 of its 39 references that are in the map. - First posted on arXiv 28 Jul 2022. ## Placed in - LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/) ## Builds on (most cited first, 21 of 21) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018) - [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019) - [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014) - [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://mlsanity.com/paper/1502.05698/) (2015) - [Transformers as Soft Reasoners over Language](https://mlsanity.com/paper/2002.05867/) (2020) - [Compositional Attention Networks for Machine Reasoning](https://mlsanity.com/paper/1803.03067/) (2018) - [End-to-End Differentiable Proving](https://mlsanity.com/paper/1705.11040/) (2017) - [Ask Me Anything: Dynamic Memory Networks for Natural Language Processing](https://mlsanity.com/paper/1506.07285/) (2015) - [Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision](https://mlsanity.com/paper/1611.00020/) (2016) - [Neural Programmer-Interpreters](https://mlsanity.com/paper/1511.06279/) (2015) - [Probing Natural Language Inference Models through Semantic Fragments](https://mlsanity.com/paper/1909.07521/) (2019) - [Dynamic Memory Networks for Visual and Textual Question Answering](https://mlsanity.com/paper/1603.01417/) (2016) - [Neural Logic Reinforcement Learning](https://mlsanity.com/paper/1904.10729/) (2019) - [AbductionRules: Training Transformers to Explain Unexpected Inputs](https://mlsanity.com/paper/2203.12186/) (2022) - [DeepLogic: Towards End-to-End Differentiable Logical Reasoning](https://mlsanity.com/paper/1805.07433/) (2018) - [HHH: An Online Medical Chatbot System based on Knowledge Graph and Hierarchical Bi-Directional Attention](https://mlsanity.com/paper/2002.03140/) (2020) - [DeepQR: Neural-based Quality Ratings for Learnersourced Multiple-Choice Questions](https://mlsanity.com/paper/2111.10058/) (2021) - [Relating Blindsight and AI: A Review](https://mlsanity.com/paper/2201.00616/) (2021) ## Built on by (most cited first, 15 of 15) - [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025) - [Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free](https://mlsanity.com/paper/2505.06708/) (2025) - [Reasoning with Language Model Prompting: A Survey](https://mlsanity.com/paper/2212.09597/) (2022) - [Natural Language Reasoning, A Survey](https://mlsanity.com/paper/2303.14725/) (2023) - [Large Language Models Are Not Strong Abstract Reasoners](https://mlsanity.com/paper/2305.19555/) (2023) - [Empowering LLMs with Logical Reasoning: A Comprehensive Survey](https://mlsanity.com/paper/2502.15652/) (2025) - [Enhancing Reasoning Capabilities of LLMs via Principled Synthetic Logic Corpus](https://mlsanity.com/paper/2411.12498/) (2024) - [Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism](https://mlsanity.com/paper/2405.15302/) (2024) - [Abstract Meaning Representation-Based Logic-Driven Data Augmentation for Logical Reasoning](https://mlsanity.com/paper/2305.12599/) (2023) - [Assessing and Enhancing the Robustness of Large Language Models with Task Structure Variations for Logical Reasoning](https://mlsanity.com/paper/2310.09430/) (2023) - [Conflict-Aware Fusion: Mitigating Logic Inertia in Large Language Models via Structured Cognitive Priors](https://mlsanity.com/paper/2512.06393/) (2025) - [LogicNet: A Logical Consistency Embedded Face Attribute Learning Network](https://mlsanity.com/paper/2311.11208/) (2023) - [Reasoning Bias of Next Token Prediction Training](https://mlsanity.com/paper/2502.02007/) (2025) - [PrologMCP: A Standardized Prolog Tool Interface for LLM Agents](https://mlsanity.com/paper/2606.14935/) (2026) - [Quantifying Logical Consistency in Transformers via Query-Key Alignment](https://mlsanity.com/paper/2502.17017/) (2025) ## Abstract (arXiv) Combining deep learning with symbolic logic reasoning aims to capitalize on the success of both fields and is drawing increasing attention. Inspired by DeepLogic, an end-to-end model trained to perform inference on logic programs, we introduce IMA-GloVe-GA, an iterative neural inference network for multi-step reasoning expressed in natural language. In our model, reasoning is performed using an iterative memory neural network based on RNN with a gated attention mechanism. We evaluate IMA-GloVe-GA on three datasets: PARARULES, CONCEPTRULES V1 and CONCEPTRULES V2. Experimental results show DeepLogic with gated attention can achieve higher test accuracy than DeepLogic and other RNN baseline models. Our model achieves better out-of-distribution generalisation than RoBERTa-Large when the rules have been shuffled. Furthermore, to address the issue of unbalanced distribution of reasoning depths in the current multi-step reasoning datasets, we develop PARARULE-Plus, a large dataset with more examples that require deeper reasoning steps. Experimental results show that the addition of PARARULE-Plus can increase the model's performance on examples requiring deeper reasoning depths. The source code and data are available at https://github.com/Strong-AI-Lab/Multi-Step-Deductive-Reasoning-Over-Natural-Language.