mlsanityData to 7 Oct 2026

ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction

arXiv 2112.01488 ↗First posted 2 Dec 2021cs.IR cs.CLcited by 464

Not on the map. Why?

Cited by 119 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Neural information retrieval (IR) has greatly advanced search and other knowledge-intensive language tasks. While many neural IR methods encode queries and documents into single-vector representations, late interaction models produce multi-vector representations at the granularity of each token and decompose relevance modeling into scalable token-level computations. This decomposition has been shown to make late interaction more effective, but it inflates the space footprint of these models by an order of magnitude. In this work, we introduce ColBERTv2, a retriever that couples an aggressive residual compression mechanism with a denoised supervision strategy to simultaneously improve the quality and space footprint of late interaction. We evaluate ColBERTv2 across a wide range of benchmarks, establishing state-of-the-art quality within and outside the training domain while reducing the space footprint of late interaction models by 6--10×.

Key facts. Cited by 464 papers in the map (data through 7 Oct 2026). Builds on 54 of its 78 references that are in the map. First posted on arXiv 2 Dec 2021.

Builds on

54 of 78 references.

References · 54

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2017Billion-scale similarity search with GPUscited by 2,477
  7. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  8. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  9. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  10. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  11. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  12. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
38 more
  1. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  2. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  3. 2019Passage Re-ranking with BERTcited by 629
  4. 2020Fact or Fiction: Verifying Scientific Claimscited by 546
  5. 2020SPECTER: Document-level Representation Learning using Citation-informed Transformerscited by 426
  6. 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
  7. 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
  8. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  9. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 213
  10. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  11. 2020HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verificationcited by 199
  12. 2020CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claimscited by 195
  13. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  14. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 164
  15. 2021Optimizing Dense Retrieval Model Training with Hard Negativescited by 154
  16. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 143
  17. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
  18. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  19. 2020Learning Dense Representations of Phrases at Scalecited by 91
  20. 2020Relevance-guided Supervision for OpenQA with ColBERTcited by 71
  21. 2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
  22. 2021Learning Passage Impacts for Inverted Indexescited by 68
  23. 2021Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrievalcited by 63
  24. 2021GooAQ: Open Question Answering with Diverse Answer Typescited by 53
  25. 2021Efficient Passage Retrieval with Hashing for Open-domain Question Answeringcited by 53
  26. 2021Domain-matched Pre-training Tasks for Dense Retrievalcited by 53
  27. 2020Distilling Dense Representations for Ranking using Tightly-Coupled Teacherscited by 52
  28. 2021PAIR: Leveraging Passage-Centric Similarity Relation for Improving Dense Passage Retrievalcited by 50
  29. 2020RepBERT: Contextualized Text Embeddings for First-Stage Retrievalcited by 46
  30. 2020Efficient Document Re-Ranking for Transformers by Precomputing Term Representationscited by 44
  31. 2021Hindsight: Posterior-guided training of retrievers for improved open-ended generationcited by 38
  32. 2021Jointly Optimizing Query Encoder and Product Quantization to Improve Retrieval Performancecited by 35
  33. 2017AdaComp : Adaptive Residual Gradient Compression for Data-Parallel Distributed Trainingcited by 33
  34. 2021Phrase Retrieval Learns Passage Retrieval, Toocited by 33
  35. 2020A Memory Efficient Baseline for Open Domain Question Answeringcited by 32
  36. 2019A Double Residual Compression Algorithm for Efficient Distributed Learningcited by 31
  37. 2021Learning Discrete Representations via Constrained Clustering for Effective and Efficient Dense Retrievalcited by 27
  38. 2021Zero-Shot Dense Retrieval with Momentum Adversarial Domain Invariant Representationscited by 27

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 42; 2023: 63; 2024: 95; 2025: 117; 2026: 144.

464 papers.

Sort:

By area:

  1. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  2. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  3. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  4. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  5. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  6. 2022Memory-Based Model Editing at Scalecited by 443
  7. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  8. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  9. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 279
  10. 2024HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Modelscited by 273
  11. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
  12. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
38 more
  1. 2023Landmark Attention: Random-Access Infinite Context Length for Transformerscited by 206
  2. 2025EmbeddingGemma: Powerful and Lightweight Text Representationscited by 128
  3. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  4. 2023How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrievalcited by 116
  5. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 116
  6. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 110
  7. 2022Task-aware Retrieval with Instructionscited by 107
  8. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  9. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 95
  10. 2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88
  11. 2023Answering Questions by Meta-Reasoning over Multiple Chains of Thoughtcited by 87
  12. 2024Gecko: Versatile Text Embeddings Distilled from Large Language Modelscited by 81
  13. 2023Context-faithful Prompting for Large Language Modelscited by 76
  14. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 69
  15. 2024SPLADE-v3: New baselines for SPLADEcited by 69
  16. 2024KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generationcited by 67
  17. 2025MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understandingcited by 66
  18. 2023Strategic Reasoning with Language Modelscited by 64
  19. 2022InPars: Data Augmentation for Information Retrieval using Large Language Modelscited by 60
  20. 2024RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generationcited by 59
  21. 2022Dialog Inpainting: Turning Documents into Dialogscited by 56
  22. 2022When Language Model Meets Private Librarycited by 54
  23. 2024STaRK: Benchmarking LLM Retrieval on Textual and Relational Knowledge Basescited by 53
  24. 2023Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answeringcited by 51
  25. 2024Generate-then-Ground in Retrieval-Augmented Generation for Multi-hop Question Answeringcited by 51
  26. 2025RAG vs. GraphRAG: A Systematic Evaluation and Key Insightscited by 51
  27. 2025Retrieval Models Aren't Tool-Savvy: Benchmarking Tool Retrieval for Large Language Modelscited by 46
  28. 2022Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformercited by 45
  29. 2024RAG-QA Arena: Evaluating Domain Robustness for Long-form Retrieval Augmented Question Answeringcited by 43
  30. 2023Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generatorscited by 42
  31. 2024Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankerscited by 42
  32. 2022ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrievalcited by 41
  33. 2024UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-world Document Analysiscited by 40
  34. 2025GFM-RAG: Graph Foundation Model for Retrieval Augmented Generationcited by 40
  35. 2022Curriculum Learning for Dense Retrieval Distillationcited by 38
  36. 2024Fine-Tuning and Prompt Optimization: Two Great Steps that Work Better Togethercited by 38
  37. 2025Parameters vs. Context: Fine-Grained Control of Knowledge Reliance in Language Modelscited by 38
  38. 2023UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankerscited by 37

50 most cited of 464 All 464 (JSON)