mlsanityData to 7 Oct 2026

Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval

arXiv 2007.00808 ↗First posted 1 Jul 2020cs.IR cs.CL cs.LGcited by 846

Not on the map. Why?

Cited by 137 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Conducting text retrieval in a dense learned representation space has many intriguing advantages over sparse retrieval. Yet the effectiveness of dense retrieval (DR) often requires combination with sparse retrieval. In this paper, we identify that the main bottleneck is in the training mechanisms, where the negative instances used in training are not representative of the irrelevant documents in testing. This paper presents Approximate nearest neighbor Negative Contrastive Estimation (ANCE), a training mechanism that constructs negatives from an Approximate Nearest Neighbor (ANN) index of the corpus, which is parallelly updated with the learning process to select more realistic negative training instances. This fundamentally resolves the discrepancy between the data distribution used in the training and testing of DR. In our experiments, ANCE boosts the BERT-Siamese DR model to outperform all competitive dense and sparse retrieval baselines. It nearly matches the accuracy of sparse-retrieval-and-BERT-reranking using dot-product in the ANCE-learned representation space and provides almost 100x speed-up.

Key facts. Cited by 846 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 84; 2022: 136; 2023: 141; 2024: 171; 2025: 170; 2026: 144.

846 papers.

Sort:

By area:

  1. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  2. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  4. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  5. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  6. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  7. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  8. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 508
  9. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  10. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  11. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  12. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 402
38 more
  1. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  2. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 352
  3. 2023Large Language Models for Information Retrieval: A Surveycited by 331
  4. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 326
  5. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 267
  6. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
  7. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  8. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  9. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
  10. 2024Tool Learning with Large Language Models: A Surveycited by 221
  11. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  12. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  13. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  14. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  15. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  16. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 166
  17. 2024FlashRAG: A Modular Toolkit for Efficient Retrieval-Augmented Generation Researchcited by 165
  18. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  19. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 159
  20. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
  21. 2023Query2doc: Query Expansion with Large Language Modelscited by 146
  22. 2023Unified Demonstration Retriever for In-Context Learningcited by 141
  23. 2021Simple Entity-Centric Questions Challenge Dense Retrieverscited by 140
  24. 2021From Canonical Correlation Analysis to Self-supervised Graph Neural Networkscited by 138
  25. 2025Memory Injection Attacks on LLM Agents via Query-Only Interactioncited by 127
  26. 2021COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretrainingcited by 126
  27. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
  28. 2021GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrievalcited by 119
  29. 2023Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answeringcited by 119
  30. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
  31. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  32. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  33. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
  34. 2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 114
  35. 2024RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrievalcited by 114
  36. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  37. 2023Evaluating Open-Domain Question Answering in the Era of Large Language Modelscited by 110
  38. 2024LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMscited by 104

50 most cited of 846 All 846 (JSON)