mlsanityData to 14 Oct 2026

Noisy Pair Corrector for Dense Retrieval

arXiv 2311.03798 ↗First posted 7 Nov 2023cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Most dense retrieval models contain an implicit assumption: the training query-document pairs are exactly matched. Since it is expensive to annotate the corpus manually, training pairs in real-world applications are usually collected automatically, which inevitably introduces mismatched-pair noise. In this paper, we explore an interesting and challenging problem in dense retrieval, how to train an effective model with mismatched-pair noise. To solve this problem, we propose a novel approach called Noisy Pair Corrector (NPC), which consists of a detection module and a correction module. The detection module estimates noise pairs by calculating the perplexity between annotated positive and easy negative documents. The correction module utilizes an exponential moving average (EMA) model to provide a soft supervised signal, aiding in mitigating the effects of noise. We conduct experiments on text-retrieval benchmarks Natural Question and TriviaQA, code-search benchmarks StaQC and SO-DS. Experimental results show that NPC achieves excellent performance in handling both synthetic and realistic noise.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 39 of its 57 references that are in the map. First posted on arXiv 7 Nov 2023.

Builds on

39 of 57 references.

References · 39

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2017Billion-scale similarity search with GPUscited by 2,497
  8. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
  9. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,059
  10. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  11. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 695
  12. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
27 more
  1. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  2. 2017Robust Loss Functions under Label Noise for Deep Neural Networkscited by 443
  3. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 413
  4. 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 361
  5. 2020Normalized Loss Functions for Deep Learning with Noisy Labelscited by 257
  6. 2020Distilling Knowledge from Reader to Retriever for Question Answeringcited by 226
  7. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  8. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 176
  9. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 172
  10. 2020Pre-training Tasks for Embedding-based Large-scale Retrievalcited by 153
  11. 2020A Survey of Label-noise Representation Learning: Past, Present and Futurecited by 122
  12. 2021Understanding and Improving Early Stopping for Learning with Noisy Labelscited by 113
  13. 2021Adversarial Retriever-Ranker for dense text retrievalcited by 83
  14. 2021End-to-End Training of Neural Retrievers for Open-Domain Question Answeringcited by 75
  15. 2021Poolingformer: Long Document Modeling with Pooling Attentioncited by 60
  16. 2022Questions Are All You Need to Train a Dense Passage Retrievercited by 47
  17. 2022Multi-View Document Representation Learning for Open-Domain Dense Retrievalcited by 41
  18. 2022Curriculum Learning for Dense Retrieval Distillationcited by 38
  19. 2018StaQC: A Systematically Mined Question-Code Dataset from Stack Overflowcited by 30
  20. 2020Is Retriever Merely an Approximator of Reader?cited by 30
  21. 2023Noisy Correspondence Learning with Meta Similarity Correctioncited by 26
  22. 2022Pre-train a Discriminative Text Encoder for Dense Retrieval via Contrastive Span Predictioncited by 23
  23. 2022UnifieR: A Unified Retriever for Large-Scale Retrievalcited by 19
  24. 2020Neural Code Search Revisited: Enhancing Code Snippet Retrieval through Natural Language Intentcited by 16
  25. 2022LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrievalcited by 15
  26. 2022Metric-guided Distillation: Distilling Knowledge from the Metric to Ranker and Retriever for Generative Commonsense Reasoningcited by 8
  27. 2022CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansioncited by 4

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

  1. 2024On the Noise Robustness of In-Context Learning for Text Generationcited by 16
  2. 2025Scientific Paper Retrieval with LLM-Guided Semantic-Based Rankingcited by 10
  3. 2024Breaking the Memory Barrier: Near Infinite Batch Size Scaling for Contrastive Losscited by 4
  4. 2025NCL-CIR: Noise-aware Contrastive Learning for Composed Image Retrievalcited by 1
  5. 2025DTKG: Dual-Track Knowledge Graph-Verified Reasoning Framework for Multi-Hop QAcited by 0