Data to 7 Oct 2026

RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering

arXiv 2010.08191 ↗First posted 16 Oct 2020cs.CL cs.IRcited by 410

Not on the map. Why?

Cited by 41 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In open-domain question answering, dense passage retrieval has become a new paradigm to retrieve relevant passages for finding answers. Typically, the dual-encoder architecture is adopted to learn dense representations of questions and passages for semantic matching. However, it is difficult to effectively train a dual-encoder due to the challenges including the discrepancy between training and inference, the existence of unlabeled positives and limited training data. To address these challenges, we propose an optimized training approach, called RocketQA, to improving dense passage retrieval. We make three major technical contributions in RocketQA, namely cross-batch negatives, denoised hard negatives and data augmentation. The experiment results show that RocketQA significantly outperforms previous state-of-the-art models on both MSMARCO and Natural Questions. We also conduct extensive experiments to examine the effectiveness of the three strategies in RocketQA. Besides, we demonstrate that the performance of end-to-end QA can be improved based on our RocketQA retriever.

Key facts. Cited by 410 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 44; 2022: 92; 2023: 92; 2024: 85; 2025: 46; 2026: 51.

410 papers.

Sort:

By area:

2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
2021Learning To Retrieve Prompts for In-Context Learningcited by 641
2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
2021Large Dual Encoders Are Generalizable Retrieverscited by 454
2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
2023Large Language Models for Information Retrieval: A Surveycited by 331

38 more

2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
2023Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Modelscited by 180
2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 171
2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
2023Query2doc: Query Expansion with Large Language Modelscited by 146
2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 146
2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
2021Scaling Deep Contrastive Learning Batch Size under Memory Limited Setupcited by 123
2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 114
2022Task-aware Retrieval with Instructionscited by 107
2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104
2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 95
2022SimKGC: Simple Contrastive Knowledge Graph Completion with Pre-trained Language Modelscited by 90
2023Improving Language Models via Plug-and-Play Retrieval Feedbackcited by 87
2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 83
2021Adversarial Retriever-Ranker for dense text retrievalcited by 82
2021The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Modelscited by 81
2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 81
2023Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrievalcited by 77
2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
2021KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answeringcited by 66
2024MindSearch: Mimicking Human Minds Elicits Deep AI Searchercited by 65
2023A Multi-Task Embedder For Retrieval Augmented LLMscited by 63
2023SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrievalcited by 61
2024Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Modelscited by 60
2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 59
2024When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentationcited by 59
2022When Language Model Meets Private Librarycited by 54

50 most cited of 410 All 410 (JSON)