mlsanityData to 7 Oct 2026

Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Survey

arXiv 2208.03197 ↗First posted 5 Aug 2022cs.CLcited by 14

Not on the map. Why?

Abstract (arXiv)

Dense retrieval (DR) approaches based on powerful pre-trained language models (PLMs) achieved significant advances and have become a key component for modern open-domain question-answering systems. However, they require large amounts of manual annotations to perform competitively, which is infeasible to scale. To address this, a growing body of research works have recently focused on improving DR performance under low-resource scenarios. These works differ in what resources they require for training and employ a diverse set of techniques. Understanding such differences is crucial for choosing the right technique under a specific low-resource scenario. To facilitate this understanding, we provide a thorough structured overview of mainstream techniques for low-resource DR. Based on their required resources, we divide the techniques into three main categories: (1) only documents are needed; (2) documents and questions are needed; and (3) documents and question-answer pairs are needed. For every technique, we introduce its general-form algorithm, highlight the open issues and pros and cons. Promising directions are outlined for future research.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 152 of its 210 references that are in the map. First posted on arXiv 5 Aug 2022.

Builds on

152 of 210 references.

References · 152

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  8. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  9. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  10. 2013Auto-Encoding Variational Bayescited by 3,980
  11. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  12. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
38 more
  1. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  2. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  3. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  4. 2017Billion-scale similarity search with GPUscited by 2,477
  5. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  6. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  7. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  8. 2020Knowledge Distillation: A Surveycited by 1,578
  9. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  10. 2015Learning Transferable Features with Deep Adaptation Networkscited by 1,323
  11. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  12. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  13. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  14. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  15. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  16. 2021GPT Understands, Toocited by 715
  17. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  18. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 691
  19. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
  20. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 597
  21. 2019ELI5: Long Form Question Answeringcited by 567
  22. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  23. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  24. 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
  25. 2022UL2: Unifying Language Learning Paradigmscited by 292
  26. 2016NIPS 2016 Tutorial: Generative Adversarial Networkscited by 284
  27. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
  28. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  29. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  30. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  31. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
  32. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 213
  33. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  34. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  35. 2021PAQ: 65 Million Probably-Asked Questions and What You Can Do With Themcited by 198
  36. 2021Muppet: Massive Multi-task Representations with Pre-Finetuningcited by 192
  37. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 184
  38. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178

50 most cited of 152 All 152 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 5; 2025: 2; 2026: 2.

14 papers.

Sort:

By area:

  1. 2023LawBench: Benchmarking Legal Knowledge of Large Language Modelscited by 111
  2. 2022A Survey for Efficient Open Domain Question Answeringcited by 20
  3. 2023xPQA: Cross-Lingual Product Question Answering across 12 Languagescited by 10
  4. 2024Assessing "Implicit" Retrieval Robustness of Large Language Modelscited by 9
  5. 2022AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentationcited by 3
  6. 2024Comparing Neighbors Together Makes it Easy: Jointly Comparing Multiple Candidates for Efficient and Effective Retrievalcited by 2
  7. 2026Beyond Global Similarity: Towards Fine-Grained, Multi-Condition Multimodal Retrievalcited by 2
  8. 2025Auto-SLURP: A Benchmark Dataset for Evaluating Multi-Agent Frameworks in Smart Personal Assistantcited by 1
  9. 2026The P-Completeness of Inverted Index Traversal: On the Complexity of Evaluating Boolean Query DAGscited by 1
  10. 2023Is Translation Helpful? An Empirical Analysis of Cross-Lingual Transfer in Low-Resource Dialog Generationcited by 0
  11. 2024Building Efficient and Effective OpenQA Systems for Low-Resource Languagescited by 0
  12. 2024Domain Adaptation of Multilingual Semantic Search -- Literature Reviewcited by 0
2 more
  1. 2024Fact Checking Beyond Training Setcited by 0
  2. 2025Empowering Global Voices: A Data-Efficient, Phoneme-Tone Adaptive Approach to High-Fidelity Speech Synthesiscited by 0