# Differentiable Reasoning on Large Knowledge Bases and Natural Language

arXiv: https://arxiv.org/abs/1912.10824 · HTML: https://mlsanity.com/paper/1912.10824/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2019.

## Built on by (most cited first, 30 of 30)

- [From Statistical Relational to Neurosymbolic Artificial Intelligence: a Survey](https://mlsanity.com/paper/2108.11451/) (2021)
- [Knowledge Graphs: Opportunities and Challenges](https://mlsanity.com/paper/2303.13948/) (2023)
- [Neuro-Symbolic Inductive Logic Programming with Logical Neural Networks](https://mlsanity.com/paper/2112.03324/) (2021)
- [Neural Multi-Hop Reasoning With Logical Rules on Biomedical Knowledge Graphs](https://mlsanity.com/paper/2103.10367/) (2021)
- [RuleBert: Teaching Soft Rules to Pre-trained Language Models](https://mlsanity.com/paper/2109.13006/) (2021)
- [Relational Memory Augmented Language Models](https://mlsanity.com/paper/2201.09680/) (2022)
- [Complex Query Answering on Eventuality Knowledge Graph with Implicit Logical Constraints](https://mlsanity.com/paper/2305.19068/) (2023)
- [Neural Compositional Rule Learning for Knowledge Graph Reasoning](https://mlsanity.com/paper/2303.03581/) (2023)
- [Knowledge Graph Reasoning with Logics and Embeddings: Survey and Perspective](https://mlsanity.com/paper/2202.07412/) (2022)
- [FF-NSL: Feed-Forward Neural-Symbolic Learner](https://mlsanity.com/paper/2106.13103/) (2021)
- [Convolutional Hypercomplex Embeddings for Link Prediction](https://mlsanity.com/paper/2106.15230/) (2021)
- [Sequential Query Encoding For Complex Query Answering on Knowledge Graphs](https://mlsanity.com/paper/2302.13114/) (2023)
- [Neural-Symbolic Recursive Machine for Systematic Generalization](https://mlsanity.com/paper/2210.01603/) (2022)
- [Learning Symbolic Rules for Reasoning in Quasi-Natural Language](https://mlsanity.com/paper/2111.12038/) (2021)
- [Neural Unification for Logic Reasoning over Natural Language](https://mlsanity.com/paper/2109.08460/) (2021)
- [R5: Rule Discovery with Reinforced and Recurrent Relational Reasoning](https://mlsanity.com/paper/2205.06454/) (2022)
- [EXPLAIN, AGREE, LEARN: Scaling Learning for Neural Probabilistic Logic](https://mlsanity.com/paper/2408.08133/) (2024)
- [Learning Language Representations with Logical Inductive Bias](https://mlsanity.com/paper/2302.09458/) (2023)
- [Dolphin: A Programmable Framework for Scalable Neurosymbolic Learning](https://mlsanity.com/paper/2410.03348/) (2024)
- [On the Capabilities of Pointer Networks for Deep Deductive Reasoning](https://mlsanity.com/paper/2106.09225/) (2021)
- [Systematic Relational Reasoning With Epistemic Graph Neural Networks](https://mlsanity.com/paper/2407.17396/) (2024)
- [Neural Theorem Provers Delineating Search Area Using RNN](https://mlsanity.com/paper/2203.06985/) (2022)
- [Relational Weight Priors in Neural Networks for Abstract Pattern Learning and Language Modelling](https://mlsanity.com/paper/2103.06198/) (2021)
- [Semantic Reasoning with Differentiable Graph Transformations](https://mlsanity.com/paper/2107.09579/) (2021)
- [RNNCTPs: A Neural Symbolic Reasoning Method Using Dynamic Knowledge Partitioning Technology](https://mlsanity.com/paper/2204.08810/) (2022)
- [Differentiable Inductive Logic Programming in High-Dimensional Space](https://mlsanity.com/paper/2208.06652/) (2022)
- [A Noise-tolerant Differentiable Learning Approach for Single Occurrence Regular Expression with Interleaving](https://mlsanity.com/paper/2212.00373/) (2022)
- [On the Aggregation of Rules for Knowledge Graph Completion](https://mlsanity.com/paper/2309.00306/) (2023)
- [Natural Language Satisfiability: Exploring the Problem Distribution and Evaluating Transformer-based Language Models](https://mlsanity.com/paper/2508.17153/) (2025)
- [Overcoming the Impedance Mismatch: A Theoretical Roadmap for Fusing Foundation Models and Knowledge Graphs](https://mlsanity.com/paper/2606.15656/) (2026)

## Abstract (arXiv)

Reasoning with knowledge expressed in natural language and Knowledge Bases (KBs) is a major challenge for Artificial Intelligence, with applications in machine reading, dialogue, and question answering. General neural architectures that jointly learn representations and transformations of text are very data-inefficient, and it is hard to analyse their reasoning process. These issues are addressed by end-to-end differentiable reasoning systems such as Neural Theorem Provers (NTPs), although they can only be used with small-scale symbolic KBs. In this paper we first propose Greedy NTPs (GNTPs), an extension to NTPs addressing their complexity and scalability limitations, thus making them applicable to real-world datasets. This result is achieved by dynamically constructing the computation graph of NTPs and including only the most promising proof paths during inference, thus obtaining orders of magnitude more efficient models. Then, we propose a novel approach for jointly reasoning over KBs and textual mentions, by embedding logic facts and natural language sentences in a shared embedding space. We show that GNTPs perform on par with NTPs at a fraction of their cost while achieving competitive link prediction results on large datasets, providing explanations for predictions, and inducing interpretable models. Source code, datasets, and supplementary material are available online at https://github.com/uclnlp/gntp.