mlsanityData to 13 Oct 2026

LogicPrpBank: A Corpus for Logical Implication and Equivalence

arXiv 2402.09609 ↗First posted 14 Feb 2024cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Logic reasoning has been critically needed in problem-solving and decision-making. Although Language Models (LMs) have demonstrated capabilities of handling multiple reasoning tasks (e.g., commonsense reasoning), their ability to reason complex mathematical problems, specifically propositional logic, remains largely underexplored. This lack of exploration can be attributed to the limited availability of annotated corpora. Here, we present a well-labeled propositional logic corpus, LogicPrpBank, containing 7093 Propositional Logic Statements (PLSs) across six mathematical subjects, to study a brand-new task of reasoning logical implication and equivalence. We benchmark LogicPrpBank with widely-used LMs to show that our corpus offers a useful resource for this challenging task and there is ample room for model improvement.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 17 of its 25 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

17 of 25 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  2. 2020Language Models are Few-Shot Learnerscited by 32,320
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,322
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,107
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,281
  8. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,978
  9. 2015A large annotated corpus for learning natural language inferencecited by 1,841
  10. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 890
  11. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 865
  12. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
5 more
  1. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 333
  2. 2022Transformer for Partial Differential Equations' Operator Learningcited by 276
  3. 2022Entailment Tree Explanations via Iterative Retrieval-Generation Reasonercited by 26
  4. 2021Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoningcited by 17
  5. 2022LogicInference: A New Dataset for Teaching Logical Inference to seq2seq Modelscited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models
  2. 2025Socrates or Smartypants: Testing Logic Reasoning Capabilities of Large Language Models with Logic Programming-based Test Oracles
  3. 2024P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains
  4. 2026LiveMathematicianBench: A Live Benchmark for Research-Level Mathematical Reasoning with Proof Sketches
  5. 2025SATBench: Benchmarking LLMs' Logical Reasoning via Automated Puzzle Generation from SAT Formulas
  6. 2025Putnam-AXIOM: A Functional and Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs
  7. 2024LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models
  8. 2024Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities