mlsanityData to 7 Oct 2026

RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking

arXiv 2110.07367 ↗First posted 14 Oct 2021cs.CLcited by 175

Not on the map. Why?

Abstract (arXiv)

In various natural language processing tasks, passage retrieval and passage re-ranking are two key procedures in finding and ranking relevant information. Since both the two procedures contribute to the final performance, it is important to jointly optimize them in order to achieve mutual improvement. In this paper, we propose a novel joint training approach for dense passage retrieval and passage re-ranking. A major contribution is that we introduce the dynamic listwise distillation, where we design a unified listwise training approach for both the retriever and the re-ranker. During the dynamic distillation, the retriever and the re-ranker can be adaptively improved according to each other's relevance information. We also propose a hybrid data augmentation strategy to construct diverse training instances for listwise training approach. Extensive experiments show the effectiveness of our approach on both MSMARCO and Natural Questions datasets. Our code is available at https://github.com/PaddlePaddle/RocketQA.

Key facts. Cited by 175 papers in the map (data through 7 Oct 2026). Builds on 38 of its 49 references that are in the map. First posted on arXiv 14 Oct 2021.

Builds on

38 of 49 references.

References · 38

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  5. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  6. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  7. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  8. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  9. 2019Passage Re-ranking with BERTcited by 629
  10. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  11. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  12. 2019ERNIE 2.0: A Continual Pre-training Framework for Language Understandingcited by 230
26 more
  1. 2017Efficient Natural Language Response Suggestion for Smart Replycited by 228
  2. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  3. 2019Document Expansion by Query Predictioncited by 225
  4. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 213
  5. 2019Multi-Stage Document Ranking with BERTcited by 200
  6. 2020Generation-Augmented Retrieval for Open-domain Question Answeringcited by 186
  7. 2020Answering Complex Open-Domain Questions with Multi-Hop Dense Retrievalcited by 160
  8. 2020Embedding-based Retrieval in Facebook Searchcited by 155
  9. 2021Optimizing Dense Retrieval Model Training with Hard Negativescited by 154
  10. 2020Pre-training Tasks for Embedding-based Large-scale Retrievalcited by 153
  11. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 143
  12. 2019Learning Dense Representations for Entity Retrievalcited by 122
  13. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  14. 2019Deeper Text Understanding for IR with Contextual Neural Language Modelingcited by 111
  15. 2019Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answeringcited by 96
  16. 2021End-to-End Training of Neural Retrievers for Open-Domain Question Answeringcited by 75
  17. 2021Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipelinecited by 65
  18. 2021PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrievalcited by 50
  19. 2019Understanding the Behaviors of BERT in Rankingcited by 49
  20. 2020RepBERT: Contextualized Text Embeddings for First-Stage Retrievalcited by 46
  21. 2020Learning-to-Rank with BERT in TF-Rankingcited by 36
  22. 2020Is Retriever Merely an Approximator of Reader?cited by 30
  23. 2021Rider: Reader-Guided Passage Reranking for Open-Domain Question Answeringcited by 28
  24. 2014An Information Retrieval Approach to Short Text Conversationcited by 18
  25. 2020Neural Retrieval for Question Answering with Cross-Attention Supervised Data Augmentationcited by 18
  26. 2020Neural Passage Retrieval with Improved Negative Contrastcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 51; 2023: 47; 2024: 41; 2025: 23; 2026: 10.

175 papers.

Sort:

By area:

  1. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  2. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  3. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  4. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  5. 2023Query2doc: Query Expansion with Large Language Modelscited by 146
  6. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 146
  7. 2024jina-embeddings-v3: Multilingual Embeddings With Task LoRAcited by 138
  8. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  9. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  10. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
  11. 2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 114
  12. 2023Evaluating Open-Domain Question Answering in the Era of Large Language Modelscited by 110
38 more
  1. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  2. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 95
  3. 2024NV-Retriever: Improving text embedding models with effective hard-negative miningcited by 83
  4. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 81
  5. 2023Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrievalcited by 77
  6. 2023Learning to Retrieve In-Context Examples for Large Language Modelscited by 60
  7. 2024When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentationcited by 59
  8. 2022Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generationcited by 47
  9. 2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
  10. 2022Multi-View Document Representation Learning for Open-Domain Dense Retrievalcited by 41
  11. 2022ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrievalcited by 41
  12. 2022Curriculum Learning for Dense Retrieval Distillationcited by 38
  13. 2024UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systemscited by 38
  14. 2022Towards Robust Ranker for Text Retrievalcited by 37
  15. 2024LLMs are Also Effective Embedding Models: An In-depth Overviewcited by 32
  16. 2022Fine-Grained Distillation for Long Document Retrievalcited by 29
  17. 2023Rethinking the Role of Token Retrieval in Multi-Vector Retrievalcited by 28
  18. 2023NevIR: Negation in Neural Information Retrievalcited by 28
  19. 2024REAR: A Relevance-Aware Retrieval-Augmented Framework for Open-Domain Question Answeringcited by 28
  20. 2022Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrievalcited by 26
  21. 2023Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrievalcited by 24
  22. 2024BASES: Large-scale Web Search User Simulation with Large Language Model based Agentscited by 24
  23. 2022CharacterBERT and Self-Teaching for Improving the Robustness of Dense Retrievers on Queries with Typoscited by 22
  24. 2024Researchy Questions: A Dataset of Multi-Perspective, Decompositional Questions for LLM Web Agentscited by 22
  25. 2024Are Large Language Models Good at Utility Judgments?cited by 22
  26. 2022A Thorough Examination on Zero-shot Dense Retrievalcited by 21
  27. 2022SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrievalcited by 21
  28. 2022PROD: Progressive Distillation for Dense Retrievalcited by 20
  29. 2022A Survey for Efficient Open Domain Question Answeringcited by 20
  30. 2024The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model designcited by 20
  31. 2022HLATR: Enhance Multi-stage Text Retrieval with Hybrid List Aware Transformer Rerankingcited by 19
  32. 2022UnifieR: A Unified Retriever for Large-Scale Retrievalcited by 19
  33. 2024CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented Analysis Generationcited by 18
  34. 2022Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddingscited by 17
  35. 2025PyLate: Flexible Training and Retrieval for Late Interaction Modelscited by 17
  36. 2022ConTextual Masked Auto-Encoder for Dense Passage Retrievalcited by 16
  37. 2025SEDM: Scalable Self-Evolving Distributed Memory for Agentscited by 16
  38. 2022LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrievalcited by 15

50 most cited of 175 All 175 (JSON)