mlsanityData to 7 Oct 2026

Pruning the Index Contents for Memory Efficient Open-Domain QA

arXiv 2102.10697 ↗First posted 21 Feb 2021cs.CL cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

This work presents a novel pipeline that demonstrates what is achievable with a combined effort of state-of-the-art approaches. Specifically, it proposes the novel R2-D2 (Rank twice, reaD twice) pipeline composed of retriever, passage reranker, extractive reader, generative reader and a simple way to combine them. Furthermore, previous work often comes with a massive index of external documents that scales in the order of tens of GiB. This work presents a simple approach for pruning the contents of a massive index such that the open-domain QA system altogether with index, OS, and library components fits into 6GiB docker image while retaining only 8% of original index contents and losing only 3% EM accuracy.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 21 Feb 2021.

Builds on

33 of 45 references.

References · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2020Longformer: The Long-Document Transformercited by 3,362
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  8. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  9. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  10. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  11. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  12. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
21 more
  1. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 385
  2. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  3. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  4. 2019Multi-Stage Document Ranking with BERTcited by 200
  5. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 198
  6. 2020Generation-Augmented Retrieval for Open-domain Question Answeringcited by 186
  7. 2019Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answeringcited by 147
  8. 2017Simple and Effective Multi-Paragraph Reading Comprehensioncited by 120
  9. 2019Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answeringcited by 96
  10. 2019A Discrete Hard EM Approach for Weakly Supervised Question Answeringcited by 78
  11. 2020Relevance-guided Supervision for OpenQA with ColBERTcited by 71
  12. 2019Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Indexcited by 64
  13. 2019Knowledge Guided Text Retrieval and Reading for Open Domain Question Answeringcited by 59
  14. 2019Understanding the Behaviors of BERT in Rankingcited by 49
  15. 2020A Memory Efficient Baseline for Open Domain Question Answeringcited by 32
  16. 2020Is Retriever Merely an Approximator of Reader?cited by 30
  17. 2021Rider: Reader-Guided Passage Reranking for Open-Domain Question Answeringcited by 28
  18. 2018Ranking Paragraphs for Improving Answer Recall in Open-Domain Question Answeringcited by 24
  19. 2020Probabilistic Assumptions Matter: Improved Models for Distantly-Supervised Document-Level Question Answeringcited by 17
  20. 2020Progressively Pretrained Dense Corpus Index for Open-Domain Question Answeringcited by 11
  21. 2020Rethinking the Objectives of Extractive Question Answeringcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 3; 2023: 0; 2024: 0; 2025: 0; 2026: 1.

6 papers.

  1. 2021KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answeringcited by 66
  2. 2022Choose Your QA Model Wisely: A Systematic Study of Generative and Extractive Readers for Question Answeringcited by 11
  3. 2021Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answeringcited by 5
  4. 2022Bridging the Training-Inference Gap for Dense Phrase Retrievalcited by 1
  5. 2022A Survey on Conversational Search and Applications in Biomedicinecited by 1
  6. 2026Do Neural Retrievers Prefer Certain Documents? Evidence of Learned Relevance Priorscited by 0