# Differentiable Reasoning over a Virtual Knowledge Base

arXiv: https://arxiv.org/abs/2002.10640 · HTML: https://mlsanity.com/paper/2002.10640/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Feb 2020.

## Built on by (most cited first, 30 of 39)

- [Pre-Trained Models: Past, Present and Future](https://mlsanity.com/paper/2106.07139/) (2021)
- [Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering](https://mlsanity.com/paper/2101.00774/) (2021)
- [A Review on Language Models as Knowledge Bases](https://mlsanity.com/paper/2204.06031/) (2022)
- [LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs](https://mlsanity.com/paper/2406.15319/) (2024)
- [Neuro-Symbolic Artificial Intelligence: Current Trends](https://mlsanity.com/paper/2105.05330/) (2021)
- [Complex Knowledge Base Question Answering: A Survey](https://mlsanity.com/paper/2108.06688/) (2021)
- [SelfExplain: A Self-Explaining Architecture for Neural Text Classifiers](https://mlsanity.com/paper/2103.12279/) (2021)
- [Mention Memory: incorporating textual knowledge into Transformers through entity mention attention](https://mlsanity.com/paper/2110.06176/) (2021)
- [Automatic Generation of Socratic Subquestions for Teaching Math Word Problems](https://mlsanity.com/paper/2211.12835/) (2022)
- [QAMPARI: An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphs](https://mlsanity.com/paper/2205.12665/) (2022)
- [Towards Data-and Knowledge-Driven Artificial Intelligence: A Survey on Neuro-Symbolic Computing](https://mlsanity.com/paper/2210.15889/) (2022)
- [SAC-KG: Exploiting Large Language Models as Skilled Automatic Constructors for Domain Knowledge Graphs](https://mlsanity.com/paper/2410.02811/) (2024)
- [Adaptive Information Seeking for Open-Domain Question Answering](https://mlsanity.com/paper/2109.06747/) (2021)
- [AdaProp: Learning Adaptive Propagation for Graph Neural Network based Knowledge Graph Reasoning](https://mlsanity.com/paper/2205.15319/) (2022)
- [Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval](https://mlsanity.com/paper/2104.05883/) (2021)
- [Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering](https://mlsanity.com/paper/2204.04581/) (2022)
- [Few-shot Reranking for Multi-hop QA via Language Model Prompting](https://mlsanity.com/paper/2205.12650/) (2022)
- [Reasoning Over Virtual Knowledge Bases With Open Predicate Relations](https://mlsanity.com/paper/2102.07043/) (2021)
- [Hybrid Autoregressive Inference for Scalable Multi-hop Explanation Regeneration](https://mlsanity.com/paper/2107.11879/) (2021)
- [UNIQORN: Unified Question Answering over RDF Knowledge Graphs and Natural Language Text](https://mlsanity.com/paper/2108.08614/) (2021)
- [Empowering Language Models with Knowledge Graph Reasoning for Question Answering](https://mlsanity.com/paper/2211.08380/) (2022)
- [Hopper: Multi-hop Transformer for Spatiotemporal Reasoning](https://mlsanity.com/paper/2103.10574/) (2021)
- [Iterative Hierarchical Attention for Answering Complex Questions over Long Documents](https://mlsanity.com/paper/2106.00200/) (2021)
- [Entity Retrieval for Answering Entity-Centric Questions](https://mlsanity.com/paper/2408.02795/) (2024)
- [ASRank: Zero-Shot Re-Ranking with Answer Scent for Document Retrieval](https://mlsanity.com/paper/2501.15245/) (2025)
- [Neural Unification for Logic Reasoning over Natural Language](https://mlsanity.com/paper/2109.08460/) (2021)
- [Relation-Guided Pre-Training for Open-Domain Question Answering](https://mlsanity.com/paper/2109.10346/) (2021)
- [A Critical Review of Inductive Logic Programming Techniques for Explainable AI](https://mlsanity.com/paper/2112.15319/) (2021)
- [Exploiting Edited Large Language Models as General Scientific Optimizers](https://mlsanity.com/paper/2503.09620/) (2025)
- [QA Is the New KR: Question-Answer Pairs as Knowledge Bases](https://mlsanity.com/paper/2207.00630/) (2022)

## Abstract (arXiv)

We consider the task of answering complex multi-hop questions using a corpus as a virtual knowledge base (KB). In particular, we describe a neural module, DrKIT, that traverses textual data like a KB, softly following paths of relations between mentions of entities in the corpus. At each step the module uses a combination of sparse-matrix TFIDF indices and a maximum inner product search (MIPS) on a special index of contextual representations of the mentions. This module is differentiable, so the full system can be trained end-to-end using gradient based methods, starting from natural language inputs. We also describe a pretraining scheme for the contextual representation encoder by generating hard negative examples using existing knowledge bases. We show that DrKIT improves accuracy by 9 points on 3-hop questions in the MetaQA dataset, cutting the gap between text-based and KB-based state-of-the-art by 70%. On HotpotQA, DrKIT leads to a 10% improvement over a BERT-based re-ranking approach to retrieving the relevant passages required to answer a question. DrKIT is also very efficient, processing 10-100x more queries per second than existing multi-hop systems.