mlsanityData to 14 Oct 2026

A Survey for Efficient Open Domain Question Answering

arXiv 2211.07886 ↗First posted 15 Nov 2022cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Open domain question answering (ODQA) is a longstanding task aimed at answering factual questions from a large knowledge corpus without any explicit evidence in natural language processing (NLP). Recent works have predominantly focused on improving the answering accuracy and achieved promising progress. However, higher accuracy often comes with more memory consumption and inference latency, which might not necessarily be efficient enough for direct deployment in the real world. Thus, a trade-off between accuracy, memory consumption and processing speed is pursued. In this paper, we provide a survey of recent advances in the efficiency of ODQA models. We walk through the ODQA models and conclude the core techniques on efficiency. Quantitative analysis on memory cost, processing speed, accuracy and overall comparison are given. We hope that this work would keep interested scholars informed of the advances and open challenges in ODQA efficiency research, and thus contribute to the further development of ODQA efficiency.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 76 of its 96 references that are in the map. First posted on arXiv 15 Nov 2022.

Builds on

76 of 96 references.

References · 76

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  8. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  10. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  11. 2017Billion-scale similarity search with GPUscited by 2,497
  12. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
38 more
  1. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  2. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,210
  3. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  4. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  5. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  6. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
  7. 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 834
  8. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  9. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  10. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  11. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
  12. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
  13. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 476
  14. 2018Gender Bias in Coreference Resolutioncited by 442
  15. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 439
  16. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  17. 2019Evaluating Gender Bias in Machine Translationcited by 249
  18. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  19. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 214
  20. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 200
  21. 2020Generation-Augmented Retrieval for Open-domain Question Answeringcited by 186
  22. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  23. 2021Block Pruning For Faster Transformerscited by 177
  24. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 176
  25. 2022Structured Pruning Learns Compact and Accurate Modelscited by 170
  26. 2015Conditional Computation in Neural Networks for faster modelscited by 167
  27. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 165
  28. 2019End-to-End Open-Domain Question Answering with BERTserinicited by 152
  29. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  30. 2020UnQovering Stereotyping Biases via Underspecified Questionscited by 126
  31. 2017HashNet: Deep Learning to Hash by Continuationcited by 122
  32. 2016A Survey on Learning to Hashcited by 119
  33. 2014Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)cited by 109
  34. 2020Learning Dense Representations of Phrases at Scalecited by 91
  35. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  36. 2020Relevance-guided Supervision for OpenQA with ColBERTcited by 71
  37. 2019Real-Time Open-Domain Question Answering with Dense-Sparse Phrase Indexcited by 64
  38. 2021Semantic Models for the First-stage Retrieval: A Comprehensive Reviewcited by 57

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 11; 2025: 1; 2026: 3.

20 papers.

Sort:

By area:

  1. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  2. 2024Retrieve, Summarize, Plan: Advancing Multi-hop Question Answering with an Iterative Approachcited by 40
  3. 2022Self-Prompting Large Language Models for Zero-Shot Open-Domain QAcited by 39
  4. 2023AutoCast++: Enhancing World Event Prediction with Zero-shot Ranking-based Context Retrievalcited by 18
  5. 2024ReLiK: Retrieve and LinK, Fast and Accurate Entity Linking and Relation Extraction on an Academic Budgetcited by 16
  6. 2024Entity Retrieval for Answering Entity-Centric Questionscited by 10
  7. 2024Should We Fine-Tune or RAG? Evaluating Different Techniques to Adapt LLMs for Dialoguecited by 7
  8. 2023Impact of time and note duration tokenizations on deep learning symbolic music modelingcited by 6
  9. 2024MASSIVE Multilingual Abstract Meaning Representation: A Dataset and Baselines for Hallucination Detectioncited by 4
  10. 2024Crafting the Path: Robust Query Rewriting for Information Retrievalcited by 4
  11. 2026STRUCTSURVEY: Structured Agentic Retrieval for Automated Survey Paper Generationcited by 4
  12. 2023Chatbots as Problem Solvers: Playing Twenty Questions with Role Reversalscited by 3
8 more
  1. 2023Hawk: An Industrial-strength Multi-label Document Classifiercited by 1
  2. 2024GOVERN: Gradient Orientation Vote Ensemble for Multi-Teacher Reinforced Distillationcited by 1
  3. 2026From Triage to Discharge: A Survey of NLP Tasks, Methods, and Open Challenges in the Emergency Departmentcited by 1
  4. 2024Building Efficient and Effective OpenQA Systems for Low-Resource Languagescited by 0
  5. 2024Unsupervised multiple choices question answering via universal corpuscited by 0
  6. 2024Leveraging Retrieval-Augmented Generation for Culturally Inclusive Hakka Chatbots: Design Insights and User Perceptionscited by 0
  7. 2025Augmenting LLM Reasoning with Dynamic Notes Writing for Complex QAcited by 0
  8. 2026Temporal Preference Optimization for Unsupervised Retrievalcited by 0