mlsanityData to 14 Oct 2026

Is Retriever Merely an Approximator of Reader?

arXiv 2010.10999 ↗First posted 21 Oct 2020cs.CLcited by 30

Not on the map. Why?

Abstract (arXiv)

The state of the art in open-domain question answering (QA) relies on an efficient retriever that drastically reduces the search space for the expensive reader. A rather overlooked question in the community is the relationship between the retriever and the reader, and in particular, if the whole purpose of the retriever is just a fast approximation for the reader. Our empirical evidence indicates that the answer is no, and that the reader and the retriever are complementary to each other even in terms of accuracy only. We make a careful conjecture that the architectural constraint of the retriever, which has been originally intended for enabling approximate search, seems to also make the model more robust in large-scale search. We then propose to distill the reader into the retriever so that the retriever absorbs the strength of the reader while keeping its own benefit. Experimental results show that our method can enhance the document recall rate as well as the end-to-end QA accuracy of off-the-shelf retrievers in open-domain QA tasks.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 13; 2022: 8; 2023: 2; 2024: 5; 2025: 2.

30 papers.

Sort:

By area:

  1. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  2. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 176
  3. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 144
  4. 2024Optimization Methods for Personalizing Large Language Models through Retrieval Augmentationcited by 99
  5. 2021Adversarial Retriever-Ranker for dense text retrievalcited by 83
  6. 2021Domain-matched Pre-training Tasks for Dense Retrievalcited by 53
  7. 2021NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learnedcited by 52
  8. 2022Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformercited by 45
  9. 2022ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrievalcited by 41
  10. 2021R2-D2: A Modular Baseline for Open-Domain Question Answeringcited by 37
  11. 2021Phrase Retrieval Learns Passage Retrieval, Toocited by 33
  12. 2021Rider: Reader-Guided Passage Reranking for Open-Domain Question Answeringcited by 28
18 more
  1. 2021Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrievalcited by 28
  2. 2024Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval-Augmented Large Language Modelscited by 23
  3. 2022PROD: Progressive Distillation for Dense Retrievalcited by 21
  4. 2022SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrievalcited by 21
  5. 2022Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Surveycited by 14
  6. 2024Learning to Rank for Multiple Retrieval-Augmented Models through Iterative Utility Maximizationcited by 14
  7. 2024Retrieval-Enhanced Machine Learning: Synthesis and Opportunitiescited by 11
  8. 2021Boosted Dense Retrievercited by 10
  9. 2021Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversationscited by 9
  10. 2024D2LLM: Decomposed and Distilled Large Language Models for Semantic Searchcited by 9
  11. 2021Pruning the Index Contents for Memory Efficient Open-Domain QAcited by 6
  12. 2023Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialogcited by 6
  13. 2021Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answeringcited by 5
  14. 2022Variational Open-Domain Question Answeringcited by 5
  15. 2023Noisy Pair Corrector for Dense Retrievalcited by 5
  16. 2025Retrieval-Augmented Generation as Noisy In-Context Learning: A Unified Theory and Risk Boundscited by 4
  17. 2025LLMDistill4Ads: Using Cross-Encoders to Distill from LLM Signals for Advertiser Keyphrase Recommendations at eBaycited by 2
  18. 2022Enhanced vectors for top-k document retrieval in Question Answeringcited by 0