mlsanityData to 7 Oct 2026

A Thorough Examination on Zero-shot Dense Retrieval

arXiv 2204.12755 ↗First posted 27 Apr 2022cs.CL cs.IRcited by 21

Not on the map. Why?

Abstract (arXiv)

Recent years have witnessed the significant advance in dense retrieval (DR) based on powerful pre-trained language models (PLM). DR models have achieved excellent performance in several benchmark datasets, while they are shown to be not as competitive as traditional sparse retrieval models (e.g., BM25) in a zero-shot retrieval setting. However, in the related literature, there still lacks a detailed and comprehensive study on zero-shot retrieval. In this paper, we present the first thorough examination of the zero-shot capability of DR models. We aim to identify the key factors and analyze how they affect zero-shot retrieval performance. In particular, we discuss the effect of several key factors related to source training set, analyze the potential bias from the target dataset, and review and compare existing zero-shot DR models. Our findings provide important evidence to better understand and develop zero-shot DR models.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 28 of its 49 references that are in the map. First posted on arXiv 27 Apr 2022.

Builds on

28 of 49 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  7. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  8. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  9. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  10. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  11. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  12. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
16 more
  1. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  2. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  3. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 426
  4. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  5. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  6. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
  7. 2019MRQA 2019 Shared Task: Evaluating Generalization in Reading Comprehensioncited by 183
  8. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  9. 2020Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Taskscited by 136
  10. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  11. 2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 59
  12. 2021PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrievalcited by 50
  13. 2020The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizescited by 48
  14. 2022Out-of-Domain Semantics to the Rescue! Zero-Shot Hybrid Retrieval Modelscited by 26
  15. 2022LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrievalcited by 24
  16. 2020Neural Passage Retrieval with Improved Negative Contrastcited by 11

Built on by

Cited by year

Citing papers per year: 2022: 7; 2023: 5; 2024: 6; 2025: 3.

21 papers.

Sort:

By area:

  1. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  2. 2024OpenTab: Advancing Large Language Models as Open-domain Table Reasonerscited by 25
  3. 2022In Defense of Cross-Encoders for Zero-Shot Retrievalcited by 21
  4. 2022COCO-DR: Combating Distribution Shifts in Zero-Shot Dense Retrieval with Contrastive and Distributionally Robust Learningcited by 19
  5. 2022No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrievalcited by 17
  6. 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspectivecited by 15
  7. 2022Low-Resource Dense Retrieval for Open-Domain Question Answering: A Comprehensive Surveycited by 14
  8. 2023BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrievalcited by 11
  9. 2023Selecting which Dense Retriever to use for Zero-Shot Searchcited by 7
  10. 2024Enhancing Question Answering for Enterprise Knowledge Bases using Large Language Modelscited by 7
  11. 2022Zero-Shot Retrieval with Search Agents and Hybrid Environmentscited by 6
  12. 2022Retrieval-based Disentangled Representation Learning with Natural Language Supervisioncited by 5
9 more
  1. 2025RGAR: Recurrence Generation-augmented Retrieval for Factual-aware Medical Question Answeringcited by 5
  2. 2024A Compressive Memory-based Retrieval Approach for Event Argument Extractioncited by 3
  3. 2024DREditor: An Time-efficient Approach for Building a Domain-specific Dense Retrieval Modelcited by 2
  4. 2025Dense Retrievers Can Fail on Simple Queries: Revealing The Granularity Dilemma of Embeddingscited by 2
  5. 2023Revisiting Sparse Retrieval for Few-shot Entity Linkingcited by 1
  6. 2023Back to Basics: A Simple Recipe for Improving Out-of-Domain Retrieval in Dense Encoderscited by 1
  7. 2025A Representation Sharpening Framework for Zero Shot Dense Retrievalcited by 1
  8. 2023Generate to Understand for Representationcited by 0
  9. 2024Domain Adaptation of Multilingual Semantic Search -- Literature Reviewcited by 0