# A Memory Efficient Baseline for Open Domain Question Answering

arXiv: https://arxiv.org/abs/2012.15156 · HTML: https://mlsanity.com/paper/2012.15156/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Dec 2020.

## Built on by (most cited first, 30 of 32)

- [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [Atlas: Few-shot Learning with Retrieval Augmented Language Models](https://mlsanity.com/paper/2208.03299/) (2022)
- [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://mlsanity.com/paper/2112.01488/) (2021)
- [PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them](https://mlsanity.com/paper/2102.07033/) (2021)
- [Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrieval](https://mlsanity.com/paper/2101.00436/) (2021)
- [Efficient Passage Retrieval with Hashing for Open-domain Question Answering](https://mlsanity.com/paper/2106.00882/) (2021)
- [NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned](https://mlsanity.com/paper/2101.00133/) (2021)
- [Multilingual Autoregressive Entity Linking](https://mlsanity.com/paper/2103.12528/) (2021)
- [Nonparametric Masked Language Modeling](https://mlsanity.com/paper/2212.01349/) (2022)
- [R2-D2: A Modular Baseline for Open-Domain Question Answering](https://mlsanity.com/paper/2109.03502/) (2021)
- [A Proposed Conceptual Framework for a Representational Approach to Information Retrieval](https://mlsanity.com/paper/2110.01529/) (2021)
- [Highly Parallel Autoregressive Entity Linking with Discriminative Correction](https://mlsanity.com/paper/2109.03792/) (2021)
- [A Survey for Efficient Open Domain Question Answering](https://mlsanity.com/paper/2211.07886/) (2022)
- [MRAG: A Modular Retrieval Framework for Time-Sensitive Question Answering](https://mlsanity.com/paper/2412.15540/) (2024)
- [Boosted Dense Retriever](https://mlsanity.com/paper/2112.07771/) (2021)
- [Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans?](https://mlsanity.com/paper/2211.12707/) (2022)
- [Encoder Adaptation of Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2110.01599/) (2021)
- [ESPN: Memory-Efficient Multi-Vector Information Retrieval](https://mlsanity.com/paper/2312.05417/) (2023)
- [Pruning the Index Contents for Memory Efficient Open-Domain QA](https://mlsanity.com/paper/2102.10697/) (2021)
- [Generative Subgraph Retrieval for Knowledge Graph-Grounded Dialog Generation](https://mlsanity.com/paper/2410.09350/) (2024)
- [Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering](https://mlsanity.com/paper/2104.07242/) (2021)
- [LeanVec: Searching vectors faster by making them fit](https://mlsanity.com/paper/2312.16335/) (2023)
- [Injecting Domain Adaptation with Learning-to-hash for Effective and Efficient Zero-shot Dense Retrieval](https://mlsanity.com/paper/2205.11498/) (2022)
- [Knowledge Base Index Compression via Dimensionality and Precision Reduction](https://mlsanity.com/paper/2204.02906/) (2022)
- [Strong and Efficient Baselines for Open Domain Conversational Question Answering](https://mlsanity.com/paper/2310.14708/) (2023)
- [MATTER: Memory-Augmented Transformer Using Heterogeneous Knowledge Sources](https://mlsanity.com/paper/2406.04670/) (2024)
- [Bridging the Training-Inference Gap for Dense Phrase Retrieval](https://mlsanity.com/paper/2210.13678/) (2022)
- [Enhanced vectors for top-k document retrieval in Question Answering](https://mlsanity.com/paper/2210.10584/) (2022)
- [Towards Better Generalization in Open-Domain Question Answering by Mitigating Context Memorization](https://mlsanity.com/paper/2404.01652/) (2024)

## Abstract (arXiv)

Recently, retrieval systems based on dense representations have led to important improvements in open-domain question answering, and related tasks. While very effective, this approach is also memory intensive, as the dense vectors for the whole knowledge source need to be kept in memory. In this paper, we study how the memory footprint of dense retriever-reader systems can be reduced. We consider three strategies to reduce the index size: dimension reduction, vector quantization and passage filtering. We evaluate our approach on two question answering benchmarks: TriviaQA and NaturalQuestions, showing that it is possible to get competitive systems using less than 6Gb of memory.