# Revealing the Importance of Semantic Retrieval for Machine Reading at Scale

arXiv: https://arxiv.org/abs/1909.08041 · HTML: https://mlsanity.com/paper/1909.08041/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2019.

## Built on by (most cited first, 30 of 50)

- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models](https://mlsanity.com/paper/2405.14831/) (2024)
- [CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving](https://mlsanity.com/paper/2310.07240/) (2023)
- [Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering](https://mlsanity.com/paper/2101.00774/) (2021)
- [ProoFVer: Natural Logic Theorem Proving for Fact Verification](https://mlsanity.com/paper/2108.11357/) (2021)
- [Multilingual Autoregressive Entity Linking](https://mlsanity.com/paper/2103.12528/) (2021)
- [TR-BERT: Dynamic Token Reduction for Accelerating BERT Inference](https://mlsanity.com/paper/2105.11618/) (2021)
- [GERE: Generative Evidence Retrieval for Fact Verification](https://mlsanity.com/paper/2204.05511/) (2022)
- [Phrase Retrieval Learns Passage Retrieval, Too](https://mlsanity.com/paper/2109.08133/) (2021)
- [Adaptive Information Seeking for Open-Domain Question Answering](https://mlsanity.com/paper/2109.06747/) (2021)
- [Multi-Step Reasoning Over Unstructured Text with Beam Dense Retrieval](https://mlsanity.com/paper/2104.05883/) (2021)
- [Highly Parallel Autoregressive Entity Linking with Discriminative Correction](https://mlsanity.com/paper/2109.03792/) (2021)
- [Modeling Multi-hop Question Answering as Single Sequence Prediction](https://mlsanity.com/paper/2205.09226/) (2022)
- [Few-shot Reranking for Multi-hop QA via Language Model Prompting](https://mlsanity.com/paper/2205.12650/) (2022)
- [Chain-of-Skills: A Configurable Model for Open-domain Question Answering](https://mlsanity.com/paper/2305.03130/) (2023)
- [Dynamic Semantic Graph Construction and Reasoning for Explainable Multi-hop Science Question Answering](https://mlsanity.com/paper/2105.11776/) (2021)
- [Topic-Aware Evidence Reasoning and Stance-Aware Aggregation for Fact Verification](https://mlsanity.com/paper/2106.01191/) (2021)
- [Exploiting Reasoning Chains for Multi-hop Science Question Answering](https://mlsanity.com/paper/2109.02905/) (2021)
- [Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge](https://mlsanity.com/paper/2210.12338/) (2022)
- [A Multi-Level Attention Model for Evidence-Based Fact Checking](https://mlsanity.com/paper/2106.00950/) (2021)
- [Robustifying Multi-hop QA through Pseudo-Evidentiality Training](https://mlsanity.com/paper/2107.03242/) (2021)
- [SUQL: Conversational Search over Structured and Unstructured Data with Large Language Models](https://mlsanity.com/paper/2311.09818/) (2023)
- [Mitigating False-Negative Contexts in Multi-document Question Answering with Retrieval Marginalization](https://mlsanity.com/paper/2103.12235/) (2021)
- [ReSel: N-ary Relation Extraction from Scientific Text and Tables by Learning to Retrieve and Select](https://mlsanity.com/paper/2210.14427/) (2022)
- [Counterfactual Multihop QA: A Cause-Effect Approach for Reducing Disconnected Reasoning](https://mlsanity.com/paper/2210.07138/) (2022)
- [Graph Reasoning with Context-Aware Linearization for Interpretable Fact Extraction and Verification](https://mlsanity.com/paper/2109.12349/) (2021)
- [Progressive Evidence Refinement for Open-domain Multimodal Retrieval Question Answering](https://mlsanity.com/paper/2310.09696/) (2023)
- [Multi-view Content-aware Indexing for Long Document Retrieval](https://mlsanity.com/paper/2404.15103/) (2024)
- [Memory Augmented Sequential Paragraph Retrieval for Multi-hop Question Answering](https://mlsanity.com/paper/2102.03741/) (2021)
- [A Simple Approach to Jointly Rank Passages and Select Relevant Sentences in the OBQA Context](https://mlsanity.com/paper/2109.10497/) (2021)

## Abstract (arXiv)

Machine Reading at Scale (MRS) is a challenging task in which a system is given an input query and is asked to produce a precise output by "reading" information from a large knowledge base. The task has gained popularity with its natural combination of information retrieval (IR) and machine comprehension (MC). Advancements in representation learning have led to separated progress in both IR and MC; however, very few studies have examined the relationship and combined design of retrieval and comprehension at different levels of granularity, for development of MRS systems. In this work, we give general guidelines on system design for MRS by proposing a simple yet effective pipeline system with special consideration on hierarchical semantic retrieval at both paragraph and sentence level, and their potential effects on the downstream task. The system is evaluated on both fact verification and open-domain multihop QA, achieving state-of-the-art results on the leaderboard test sets of both FEVER and HOTPOTQA. To further demonstrate the importance of semantic retrieval, we present ablation and analysis studies to quantify the contribution of neural retrieval modules at both paragraph-level and sentence-level, and illustrate that intermediate semantic retrieval modules are vital for not only effectively filtering upstream information and thus saving downstream computation, but also for shaping upstream data distribution and providing better data for downstream modeling. Code/data made publicly available at: https://github.com/easonnie/semanticRetrievalMRS