Can Neural Networks Understand Logical Entailment?
Not on the map. Why?
Abstract (arXiv)
We introduce a new dataset of logical entailments for the purpose of measuring models' ability to capture and exploit the structure of logical expressions against an entailment prediction task. We use this task to compare a series of architectures which are ubiquitous in the sequence-processing literature, in addition to a new model class---PossibleWorldNets---which computes entailment as a "convolution over possible worlds". Results show that convolutional networks present the wrong inductive bias for this class of problems relative to LSTM RNNs, tree-structured neural networks outperform LSTM RNNs due to their enhanced ability to exploit the syntax of logic, and PossibleWorldNets outperform all benchmarks.
Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Feb 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 9; 2022: 3; 2023: 3; 2024: 3; 2025: 4; 2026: 2.
24 papers.
Sort:
By area:
- 2021Proof Artifact Co-training for Theorem Proving with Language Modelscited by 130
- 2021Machine Learning for Electronic Design Automation: A Surveycited by 71
- 2021LIME: Learning Inductive Bias for Primitives of Mathematical Reasoningcited by 46
- 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Modelscited by 35
- 2024Logically Consistent Language Models via Neuro-Symbolic Integrationcited by 30
- 2021Pushing the Limits of Rule Reasoning in Transformers through Natural Language Satisfiabilitycited by 18
- 2023Symbol-LLM: Leverage Language Models for Symbolic System in Visual Human Activity Reasoningcited by 16
- 2021Recognizing and Verifying Mathematical Equations using Multiplicative Differential Neural Unitscited by 8
- 2022Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarkscited by 8
- 2024Learning Guided Automated Reasoning: A Brief Surveycited by 7
- 2023MizAR 60 for Mizar 50cited by 6
- 2021Generating Symbolic Reasoning Problems with Transformer GANscited by 5
12 more
- 2021Neuro-Symbolic VQA: A review from the perspective of AGI desideratacited by 4
- 2021On the Capabilities of Pointer Networks for Deep Deductive Reasoningcited by 4
- 2025Giving AI Personalities Leads to More Human-Like Reasoningcited by 3
- 2026NL2LOGIC: AST-Guided Translation of Natural Language into First-Order Logic with Large Language Modelscited by 3
- 2024BAIT: Benchmarking (Embedding) Architectures for Interactive Theorem-Provingcited by 2
- 2022Embeddings as Epistemic States: Limitations on the Use of Pooling Operators for Accumulating Knowledgecited by 1
- 2026Zero-Shot Instruction Following in RL via Structured LTL Representationscited by 1
- 2021Learning Heuristics for Template-based CEGIS of Loop Invariants with Reinforcement Learningcited by 0
- 2022Extending Logical Neural Networks using First-Order Theoriescited by 0
- 2025Structured Relational Representationscited by 0
- 2025Natural Language Satisfiability: Exploring the Problem Distribution and Evaluating Transformer-based Language Modelscited by 0
- 2025Zero-Shot Instruction Following in RL via Structured LTL Representationscited by 0
- 2026Zero-Shot Instruction Following in RL via Structured LTL Representations15 Feb 2026
- 2026NL2LOGIC: AST-Guided Translation of Natural Language into First-Order Logic with Large Language Models29 Jan 2026
- 2025Zero-Shot Instruction Following in RL via Structured LTL Representations2 Dec 2025
- 2025Natural Language Satisfiability: Exploring the Problem Distribution and Evaluating Transformer-based Language Models23 Aug 2025
- 2025Structured Relational Representations17 May 2025
- 2025Giving AI Personalities Leads to More Human-Like Reasoning19 Feb 2025
- 2024Logically Consistent Language Models via Neuro-Symbolic Integration9 Sep 2024
- 2024Learning Guided Automated Reasoning: A Brief Survey6 Mar 2024
- 2024BAIT: Benchmarking (Embedding) Architectures for Interactive Theorem-Proving6 Mar 2024
- 2023Symbol-LLM: Leverage Language Models for Symbolic System in Visual Human Activity Reasoning29 Nov 2023
- 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models2 Oct 2023
- 2023MizAR 60 for Mizar 5012 Mar 2023
12 more
- 2022Embeddings as Epistemic States: Limitations on the Use of Pooling Operators for Accumulating Knowledge11 Oct 2022
- 2022Extending Logical Neural Networks using First-Order Theories6 Jul 2022
- 2022Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks22 Apr 2022
- 2021Pushing the Limits of Rule Reasoning in Transformers through Natural Language Satisfiability16 Dec 2021
- 2021Generating Symbolic Reasoning Problems with Transformer GANs19 Oct 2021
- 2021Learning Heuristics for Template-based CEGIS of Loop Invariants with Reinforcement Learning16 Jul 2021
- 2021On the Capabilities of Pointer Networks for Deep Deductive Reasoning17 Jun 2021
- 2021Neuro-Symbolic VQA: A review from the perspective of AGI desiderata13 Apr 2021
- 2021Recognizing and Verifying Mathematical Equations using Multiplicative Differential Neural Units7 Apr 2021
- 2021Proof Artifact Co-training for Theorem Proving with Language Models11 Feb 2021
- 2021LIME: Learning Inductive Bias for Primitives of Mathematical Reasoning15 Jan 2021
- 2021Machine Learning for Electronic Design Automation: A Survey10 Jan 2021
- 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Modelscited by 35
- 2023Symbol-LLM: Leverage Language Models for Symbolic System in Visual Human Activity Reasoningcited by 16
- 2024Learning Guided Automated Reasoning: A Brief Surveycited by 7
- 2025Giving AI Personalities Leads to More Human-Like Reasoningcited by 3
- 2022Extending Logical Neural Networks using First-Order Theoriescited by 0
- 2025Natural Language Satisfiability: Exploring the Problem Distribution and Evaluating Transformer-based Language Modelscited by 0
- 2025Natural Language Satisfiability: Exploring the Problem Distribution and Evaluating Transformer-based Language Models23 Aug 2025
- 2025Giving AI Personalities Leads to More Human-Like Reasoning19 Feb 2025
- 2024Learning Guided Automated Reasoning: A Brief Survey6 Mar 2024
- 2023Symbol-LLM: Leverage Language Models for Symbolic System in Visual Human Activity Reasoning29 Nov 2023
- 2023Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical Reasoning Capabilities of Language Models2 Oct 2023
- 2022Extending Logical Neural Networks using First-Order Theories6 Jul 2022
- 2021Machine Learning for Electronic Design Automation: A Surveycited by 71
- 2021LIME: Learning Inductive Bias for Primitives of Mathematical Reasoningcited by 46
- 2024Logically Consistent Language Models via Neuro-Symbolic Integrationcited by 30
- 2021Pushing the Limits of Rule Reasoning in Transformers through Natural Language Satisfiabilitycited by 18
- 2021Recognizing and Verifying Mathematical Equations using Multiplicative Differential Neural Unitscited by 8
- 2022Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarkscited by 8
- 2023MizAR 60 for Mizar 50cited by 6
- 2021Generating Symbolic Reasoning Problems with Transformer GANscited by 5
- 2021Neuro-Symbolic VQA: A review from the perspective of AGI desideratacited by 4
- 2021On the Capabilities of Pointer Networks for Deep Deductive Reasoningcited by 4
- 2026NL2LOGIC: AST-Guided Translation of Natural Language into First-Order Logic with Large Language Modelscited by 3
- 2024BAIT: Benchmarking (Embedding) Architectures for Interactive Theorem-Provingcited by 2
12 most cited of 17 All 17 (JSON)
- 2026Zero-Shot Instruction Following in RL via Structured LTL Representations15 Feb 2026
- 2026NL2LOGIC: AST-Guided Translation of Natural Language into First-Order Logic with Large Language Models29 Jan 2026
- 2025Zero-Shot Instruction Following in RL via Structured LTL Representations2 Dec 2025
- 2025Structured Relational Representations17 May 2025
- 2024Logically Consistent Language Models via Neuro-Symbolic Integration9 Sep 2024
- 2024BAIT: Benchmarking (Embedding) Architectures for Interactive Theorem-Proving6 Mar 2024
- 2023MizAR 60 for Mizar 5012 Mar 2023
- 2022Embeddings as Epistemic States: Limitations on the Use of Pooling Operators for Accumulating Knowledge11 Oct 2022
- 2022Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks22 Apr 2022
- 2021Pushing the Limits of Rule Reasoning in Transformers through Natural Language Satisfiability16 Dec 2021
- 2021Generating Symbolic Reasoning Problems with Transformer GANs19 Oct 2021
- 2021Learning Heuristics for Template-based CEGIS of Loop Invariants with Reinforcement Learning16 Jul 2021
12 newest of 17 All 17 (JSON)