# Is Retriever Merely an Approximator of Reader?

arXiv: https://arxiv.org/abs/2010.10999 · HTML: https://mlsanity.com/paper/2010.10999/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2020.

## Built on by (most cited first, 30 of 30)

- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking](https://mlsanity.com/paper/2110.07367/) (2021)
- [Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidence](https://mlsanity.com/paper/2210.13701/) (2022)
- [Optimization Methods for Personalizing Large Language Models through Retrieval Augmentation](https://mlsanity.com/paper/2404.05970/) (2024)
- [Adversarial Retriever-Ranker for dense text retrieval](https://mlsanity.com/paper/2110.03611/) (2021)
- [Domain-matched Pre-training Tasks for Dense Retrieval](https://mlsanity.com/paper/2107.13602/) (2021)
- [NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned](https://mlsanity.com/paper/2101.00133/) (2021)
- [Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer](https://mlsanity.com/paper/2212.02027/) (2022)
- [ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrieval](https://mlsanity.com/paper/2205.09153/) (2022)
- [R2-D2: A Modular Baseline for Open-Domain Question Answering](https://mlsanity.com/paper/2109.03502/) (2021)
- [Phrase Retrieval Learns Passage Retrieval, Too](https://mlsanity.com/paper/2109.08133/) (2021)
- [Rider: Reader-Guided Passage Reranking for Open-Domain Question Answering](https://mlsanity.com/paper/2101.00294/) (2021)
- [Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval](https://mlsanity.com/paper/2103.11920/) (2021)
- [Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2405.00175/) (2024)
- [PROD: Progressive Distillation for Dense Retrieval](https://mlsanity.com/paper/2209.13335/) (2022)
- [SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval](https://mlsanity.com/paper/2210.11773/) (2022)
- [Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Survey](https://mlsanity.com/paper/2208.03197/) (2022)
- [Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximization](https://mlsanity.com/paper/2410.09942/) (2024)
- [Retrieval-Enhanced Machine Learning: Synthesis and Opportunities](https://mlsanity.com/paper/2407.12982/) (2024)
- [Boosted Dense Retriever](https://mlsanity.com/paper/2112.07771/) (2021)
- [Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations](https://mlsanity.com/paper/2109.13087/) (2021)
- [D2LLM: Decomposed and Distilled Large Language Models for Semantic Search](https://mlsanity.com/paper/2406.17262/) (2024)
- [Pruning the Index Contents for Memory Efficient Open-Domain QA](https://mlsanity.com/paper/2102.10697/) (2021)
- [Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialog](https://mlsanity.com/paper/2305.10149/) (2023)
- [Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering](https://mlsanity.com/paper/2104.07242/) (2021)
- [Variational Open-Domain Question Answering](https://mlsanity.com/paper/2210.06345/) (2022)
- [Noisy Pair Corrector for Dense Retrieval](https://mlsanity.com/paper/2311.03798/) (2023)
- [Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Bounds](https://mlsanity.com/paper/2506.03100/) (2025)
- [LLMDistill4Ads: Using Cross-Encoders to Distill from LLM Signals for Advertiser Keyphrase Recommendations at eBay](https://mlsanity.com/paper/2508.03628/) (2025)
- [Enhanced vectors for top-k document retrieval in Question Answering](https://mlsanity.com/paper/2210.10584/) (2022)

## Abstract (arXiv)

The state of the art in open-domain question answering (QA) relies on an efficient retriever that drastically reduces the search space for the expensive reader. A rather overlooked question in the community is the relationship between the retriever and the reader, and in particular, if the whole purpose of the retriever is just a fast approximation for the reader. Our empirical evidence indicates that the answer is no, and that the reader and the retriever are complementary to each other even in terms of accuracy only. We make a careful conjecture that the architectural constraint of the retriever, which has been originally intended for enabling approximate search, seems to also make the model more robust in large-scale search. We then propose to distill the reader into the retriever so that the retriever absorbs the strength of the reader while keeping its own benefit. Experimental results show that our method can enhance the document recall rate as well as the end-to-end QA accuracy of off-the-shelf retrievers in open-domain QA tasks.