# Learning Open Domain Multi-hop Search Using Reinforcement Learning

arXiv: https://arxiv.org/abs/2205.15281 · HTML: https://mlsanity.com/paper/2205.15281/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 20 references that are in the map.
- First posted on arXiv 30 May 2022.

## Builds on (most cited first, 8 of 8)

- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/1902.07669/) (2019)
- [Constructing Datasets for Multi-hop Reading Comprehension Across Documents](https://mlsanity.com/paper/1710.06481/) (2017)
- [Dual Learning for Machine Translation](https://mlsanity.com/paper/1611.00179/) (2016)
- [End-to-End Task-Completion Neural Dialogue Systems](https://mlsanity.com/paper/1703.01008/) (2017)
- [Improving Coreference Resolution by Learning Entity-Level Distributed Representations](https://mlsanity.com/paper/1606.01323/) (2016)
- [Learning what to read: Focused machine reading](https://mlsanity.com/paper/1709.00149/) (2017)

## Abstract (arXiv)

We propose a method to teach an automated agent to learn how to search for multi-hop paths of relations between entities in an open domain. The method learns a policy for directing existing information retrieval and machine reading resources to focus on relevant regions of a corpus. The approach formulates the learning problem as a Markov decision process with a state representation that encodes the dynamics of the search process and a reward structure that minimizes the number of documents that must be processed while still finding multi-hop paths. We implement the method in an actor-critic reinforcement learning algorithm and evaluate it on a dataset of search problems derived from a subset of English Wikipedia. The algorithm finds a family of policies that succeeds in extracting the desired information while processing fewer documents compared to several baseline heuristic algorithms.