# Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval

arXiv: https://arxiv.org/abs/2104.05883 · HTML: https://mlsanity.com/paper/2104.05883/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 13 Apr 2021.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [TVQA: Localized, Compositional Video Question Answering](https://mlsanity.com/paper/1809.01696/) (2018)
- [The Web as a Knowledge-base for Answering Complex Questions](https://mlsanity.com/paper/1803.06643/) (2018)
- [AmbigQA: Answering Ambiguous Open-domain Questions](https://mlsanity.com/paper/2004.10645/) (2020)
- [Variational Reasoning for Question Answering with Knowledge Graph](https://mlsanity.com/paper/1709.04071/) (2017)
- [Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval](https://mlsanity.com/paper/2009.12756/) (2020)
- [Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering](https://mlsanity.com/paper/1911.10470/) (2019)
- [Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)](https://mlsanity.com/paper/1405.5869/) (2014)
- [Answering Complex Open-domain Questions Through Iterative Query Generation](https://mlsanity.com/paper/1910.07000/) (2019)
- [Revealing the Importance of Semantic Retrieval for Machine Reading at Scale](https://mlsanity.com/paper/1909.08041/) (2019)
- [Differentiable Reasoning over a Virtual Knowledge Base](https://mlsanity.com/paper/2002.10640/) (2020)
- [Complex Factoid Question Answering with a Free-Text Knowledge Graph](https://mlsanity.com/paper/2103.12876/) (2021)

## Built on by (most cited first, 25 of 25)

- [A Dataset for Answering Time-Sensitive Questions](https://mlsanity.com/paper/2108.06314/) (2021)
- [Generating Natural Language Proofs with Verifier-Guided Search](https://mlsanity.com/paper/2205.12443/) (2022)
- [Mention Memory: incorporating textual knowledge into Transformers through entity mention attention](https://mlsanity.com/paper/2110.06176/) (2021)
- [End-to-End Beam Retrieval for Multi-Hop Question Answering](https://mlsanity.com/paper/2308.08973/) (2023)
- [Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner](https://mlsanity.com/paper/2205.09224/) (2022)
- [What's in a Name? Answer Equivalence For Open-Domain Question Answering](https://mlsanity.com/paper/2109.05289/) (2021)
- [Few-shot Reranking for Multi-hop QA via Language Model Prompting](https://mlsanity.com/paper/2205.12650/) (2022)
- [Hybrid Autoregressive Inference for Scalable Multi-hop Explanation Regeneration](https://mlsanity.com/paper/2107.11879/) (2021)
- [UNIQORN: Unified Question Answering over RDF Knowledge Graphs and Natural Language Text](https://mlsanity.com/paper/2108.08614/) (2021)
- [Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge](https://mlsanity.com/paper/2210.12338/) (2022)
- [Reverse Question Answering: Can an LLM Write a Question so Hard (or Bad) that it Can't Answer?](https://mlsanity.com/paper/2410.15512/) (2024)
- [KiRAG: Knowledge-Driven Iterative Retriever for Enhancing Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.18397/) (2025)
- [Distantly-Supervised Evidence Retrieval Enables Question Answering without Evidence Annotation](https://mlsanity.com/paper/2110.04889/) (2021)
- [How Many Answers Should I Give? An Empirical Study of Multi-Answer Reading Comprehension](https://mlsanity.com/paper/2306.00435/) (2023)
- [ReasonChainQA: Text-based Complex Question Answering with Explainable Evidence Chains](https://mlsanity.com/paper/2210.08763/) (2022)
- [Extract, Integrate, Compete: Towards Verification Style Reading Comprehension](https://mlsanity.com/paper/2109.05149/) (2021)
- [Better Retrieval May Not Lead to Better Question Answering](https://mlsanity.com/paper/2205.03685/) (2022)
- [Interactive Machine Comprehension with Dynamic Knowledge Graphs](https://mlsanity.com/paper/2109.00077/) (2021)
- [Reimagining Retrieval Augmented Language Models for Answering Queries](https://mlsanity.com/paper/2306.01061/) (2023)
- [Gumbel Reranking: Differentiable End-to-End Reranker Optimization](https://mlsanity.com/paper/2502.11116/) (2025)
- [CIRAG: Construction-Integration Retrieval and Adaptive Generation for Multi-hop Question Answering](https://mlsanity.com/paper/2601.06799/) (2026)
- [Large Language Models Still Face Challenges in Multi-Hop Reasoning with External Knowledge](https://mlsanity.com/paper/2412.08317/) (2024)
- [Momentum Posterior Regularization for Multi-hop Dense Retrieval](https://mlsanity.com/paper/2502.20399/) (2024)
- [Cheater's Bowl: Human vs. Computer Search Strategies for Open-Domain Question Answering](https://mlsanity.com/paper/2212.03296/) (2022)
- [Forgotten History or Test-of-Time? Retrospect and Prospect on RAG from an IR Perspective](https://mlsanity.com/paper/2608.08445/) (2026)

## Abstract (arXiv)

Complex question answering often requires finding a reasoning chain that consists of multiple evidence pieces. Current approaches incorporate the strengths of structured knowledge and unstructured text, assuming text corpora is semi-structured. Building on dense retrieval methods, we propose a new multi-step retrieval approach (BeamDR) that iteratively forms an evidence chain through beam search in dense representations. When evaluated on multi-hop question answering, BeamDR is competitive to state-of-the-art systems, without using any semi-structured information. Through query composition in dense space, BeamDR captures the implicit relationships between evidence in the reasoning chain. The code is available at https://github.com/ henryzhao5852/BeamDR.