mlsanityData to 14 Oct 2026

LogicInference: A New Dataset for Teaching Logical Inference to seq2seq Models

arXiv 2203.15099 ↗First posted 28 Mar 2022cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

Machine learning models such as Transformers or LSTMs struggle with tasks that are compositional in nature such as those involving reasoning/inference. Although many datasets exist to evaluate compositional generalization, when it comes to evaluating inference abilities, options are more limited. This paper presents LogicInference, a new dataset to evaluate the ability of models to perform logical inference. The dataset focuses on inference using propositional logic and a small subset of first-order logic, represented both in semi-formal logical notation, as well as in natural language. We also report initial results using a collection of machine learning models to establish an initial baseline in this dataset.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 15 of its 25 references that are in the map. First posted on arXiv 28 Mar 2022.

Builds on

15 of 25 references.

References · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  5. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 905
  6. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  7. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 335
  8. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 284
  9. 2019Compositionality decomposed: how do neural networks generalise?cited by 265
  10. 2019CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Textcited by 162
  11. 2021Explaining Answers with Entailment Treescited by 152
  12. 2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
3 more
  1. 2020Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architecturescited by 95
  2. 2021Making Transformers Solve Compositional Taskscited by 72
  3. 2018Memorize or generalize? Searching for a compositional RNN in a haystackcited by 41

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 3; 2025: 3; 2026: 2.

12 papers.

  1. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 202
  2. 2023Natural Language Reasoning, A Surveycited by 61
  3. 2023LogiCoT: Logical Chain-of-Thought Instruction-Tuningcited by 31
  4. 2024Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanismcited by 15
  5. 2024Reason from Fallacy: Enhancing Large Language Models' Logical Reasoning through Logical Fallacy Understandingcited by 12
  6. 2025Reasoning Bias of Next Token Prediction Trainingcited by 3
  7. 2025Logical forms complement probability in understanding language model (and human) performancecited by 3
  8. 2025LogicLearner: A Tool for the Guided Practice of Propositional Logic Proofscited by 2
  9. 2023Transformers in the Service of Description Logic-based Contextscited by 0
  10. 2024LogicPrpBank: A Corpus for Logical Implication and Equivalencecited by 0
  11. 2026Stress-Testing the Reasoning Competence of LLMs With Proofs Under Minimal Formalismcited by 0
  12. 2026ChLogic: Evaluating Robustness of Logical Reasoning in Chinese Expressionscited by 0