Data to 7 Oct 2026

How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval

arXiv 2302.07452 ↗First posted 15 Feb 2023cs.IR cs.CLcited by 116

Not on the map. Why?

Cited by 43 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Various techniques have been developed in recent years to improve dense retrieval (DR), such as unsupervised contrastive learning and pseudo-query generation. Existing DRs, however, often suffer from effectiveness tradeoffs between supervised and zero-shot retrieval, which some argue was due to the limited model capacity. We contradict this hypothesis and show that a generalizable DR can be trained to achieve high accuracy in both supervised and zero-shot retrieval without increasing model size. In particular, we systematically examine the contrastive learning of DRs, under the framework of Data Augmentation (DA). Our study shows that common DA practices such as query augmentation with generative models and pseudo-relevance label creation using a cross-encoder, are often inefficient and sub-optimal. We hence propose a new DA approach with diverse queries and sources of supervision to progressively train a generalizable DR. As a result, DRAGON, our dense retriever trained with diverse augmentation, is the first BERT-base-sized DR to achieve state-of-the-art effectiveness in both supervised and zero-shot evaluations and even competes with models using more complex late interaction (ColBERTv2 and SPLADE++).

Key facts. Cited by 116 papers in the map (data through 7 Oct 2026). Builds on 43 of its 58 references that are in the map. First posted on arXiv 15 Feb 2023.

Builds on

43 of 58 references.

References · 43

2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
2015Distilling the Knowledge in a Neural Networkcited by 10,076
2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
2017Billion-scale similarity search with GPUscited by 2,477
2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940

31 more

2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
2021Large Dual Encoders Are Generalizable Retrieverscited by 454
2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
2020Overview of the TREC 2019 deep learning trackcited by 282
2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
2021Overview of the TREC 2020 deep learning trackcited by 193
2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
2020Pre-training Tasks for Embedding-based Large-scale Retrievalcited by 153
2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 143
2020Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Taskscited by 136
2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
2022Task-aware Retrieval with Instructionscited by 107
2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 95
2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 59
2021Domain-matched Pre-training Tasks for Dense Retrievalcited by 53
2022Curriculum Learning for Dense Retrieval Distillationcited by 38
2021A Replication Study of Dense Passage Retrievercited by 37
2021Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representationscited by 27
2022PROD: Progressive Distillation for Dense Retrievalcited by 20
2022ConTextual Masked Auto-Encoder for Dense Passage Retrievalcited by 16
2021Wacky Weights in Learned Sparse Representations and the Revenge of Score-at-a-Time Query Evaluationcited by 15
2021Deep Bregman Divergence for Contrastive Learning of Visual Representationscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 48; 2025: 34; 2026: 19.

116 papers.

Sort:

By area:

2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
2024The Faiss librarycited by 370
2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 201
2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 192
2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 159
2024xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Tokencited by 123
2024Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approachcited by 117
2023Retrieval meets Long Context Large Language Modelscited by 114
2024ChatQA: Surpassing GPT-4 on Conversational QA and RAGcited by 90
2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88

38 more

2024A Survey on Retrieval-Augmented Text Generation for Large Language Modelscited by 77
2024Reliable, Adaptable, and Attributable Language Models with Retrievalcited by 73
2024Scaling Retrieval-Based Language Models with a Trillion-Token Datastorecited by 54
2023InstructRetro: Instruction Tuning post Retrieval-Augmented Pretrainingcited by 53
2024ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilitiescited by 52
2024FLAME: Factuality-Aware Alignment for Large Language Modelscited by 46
2023Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Rankingcited by 44
2024BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrieverscited by 38
2024RAR-b: Reasoning as Retrieval Benchmarkcited by 33
2024BERGEN: A Benchmarking Library for Retrieval-Augmented Generationcited by 31
2023PDFTriage: Question Answering over Long, Structured Documentscited by 29
2024SimRAG: Self-Improving Retrieval-Augmented Generation for Adapting Large Language Models to Specialized Domainscited by 28
2024Nearest Neighbor Speculative Decoding for LLM Generation and Attributioncited by 26
2024RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processingcited by 25
2023Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrievalcited by 24
2024RAM-EHR: Retrieval Augmentation Meets Clinical Predictions on Electronic Health Recordscited by 21
2025PeerQA: A Scientific Question Answering Dataset from Peer Reviewscited by 21
2025DRAMA: Diverse Augmentation from Large Language Models to Smaller Dense Retrieverscited by 19
2025Collab-RAG: Boosting Retrieval-Augmented Generation for Complex Question Answering via White-Box and Black-Box LLM Collaborationcited by 19
2023GAR-meets-RAG Paradigm for Zero-Shot Information Retrievalcited by 18
2024Cocktail: A Comprehensive Information Retrieval Benchmark with LLM-Generated Documents Integrationcited by 17
2025GraphEval: A Lightweight Graph-Based LLM Framework for Idea Evaluationcited by 17
2024Generalizing Conversational Dense Retrieval via LLM-Cognition Data Augmentationcited by 16
2024TRACE the Evidence: Constructing Knowledge-Grounded Reasoning Chains for Retrieval-Augmented Generationcited by 16
2025Visual-RAG: Benchmarking Text-to-Image Retrieval Augmented Generation for Visual Knowledge Intensive Queriescited by 16
2025Collapse of Dense Retrievers: Short, Early, and Literal Biases Outranking Factual Evidencecited by 16
2026Learning Personalized Agents from Human Feedbackcited by 16
2024UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generationcited by 14
2025Graph World Modelcited by 14
2024RuleRAG: Rule-Guided Retrieval-Augmented Generation with Language Models for Question Answeringcited by 13
2023Soft Prompt Tuning for Augmenting Dense Retrieval with Large Language Modelscited by 12
2024Dense Passage Retrieval: Is it Retrieving?cited by 12
2024RouterRetriever: Routing over a Mixture of Expert Embedding Modelscited by 12
2024Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphscited by 12
2025Unsupervised Corpus Poisoning Attacks in Continuous Space for Dense Retrievalcited by 11
2024ARL2: Aligning Retrievers for Black-box Large Language Models via Self-guided Adaptive Relevance Labelingcited by 10
2024Attribute or Abstain: Large Language Models as Long Document Assistantscited by 10
2025Memory-QA: Answering Recall Questions Based on Multimodal Memoriescited by 10

50 most cited of 116 All 116 (JSON)