mlsanityData to 7 Oct 2026

ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT

arXiv 2004.12832 ↗First posted 27 Apr 2020cs.IR cs.CLcited by 1,160

Not on the map. Why?

Cited by 226 in Retrieval-augmented generation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent progress in Natural Language Understanding (NLU) is driving fast-paced advances in Information Retrieval (IR), largely owed to fine-tuning deep language models (LMs) for document ranking. While remarkably effective, the ranking models based on these LMs increase computational cost by orders of magnitude over prior approaches, particularly as they must feed each query-document pair through a massive neural network to compute a single relevance score. To tackle this, we present ColBERT, a novel ranking model that adapts deep LMs (in particular, BERT) for efficient retrieval. ColBERT introduces a late interaction architecture that independently encodes the query and the document using BERT and then employs a cheap yet powerful interaction step that models their fine-grained similarity. By delaying and yet retaining this fine-granular interaction, ColBERT can leverage the expressiveness of deep LMs while simultaneously gaining the ability to pre-compute document representations offline, considerably speeding up query processing. Beyond reducing the cost of re-ranking the documents retrieved by a traditional model, ColBERT's pruning-friendly interaction mechanism enables leveraging vector-similarity indexes for end-to-end retrieval directly from a large document collection. We extensively evaluate ColBERT using two recent passage search datasets. Results show that ColBERT's effectiveness is competitive with existing BERT-based models (and outperforms every non-BERT baseline), while executing two orders-of-magnitude faster and requiring four orders-of-magnitude fewer FLOPs per query.

Key facts. Cited by 1,160 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 74; 2022: 121; 2023: 146; 2024: 175; 2025: 265; 2026: 379.

1,160 papers.

Sort:

By area:

  1. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  2. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  3. 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,216
  4. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  5. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  6. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  7. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  8. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  9. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  10. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  11. 2021Large Dual Encoders Are Generalizable Retrieverscited by 454
  12. 2024A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Modelscited by 450
38 more
  1. 2022Memory-Based Model Editing at Scalecited by 443
  2. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 401
  3. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  4. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  5. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  6. 2022Generate rather than Retrieve: Large Language Models are Strong Context Generatorscited by 296
  7. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 279
  8. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 275
  9. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  10. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  11. 2022SGPT: GPT Sentence Embeddings for Semantic Searchcited by 204
  12. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  13. 2023In-context Autoencoder for Context Compression in a Large Language Modelcited by 190
  14. 2024ColPali: Efficient Document Retrieval with Vision Language Modelscited by 190
  15. 2023MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrievalcited by 182
  16. 2021Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answeringcited by 178
  17. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  18. 2023Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approachcited by 168
  19. 2024Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generationcited by 158
  20. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
  21. 2024Evaluation of Retrieval-Augmented Generation: A Surveycited by 146
  22. 2021End-to-End Training of Multi-Document Reader and Retriever for Open-Domain Question Answeringcited by 133
  23. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
  24. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
  25. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  26. 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 116
  27. 2022RankT5: Fine-Tuning T5 for Text Ranking with Ranking Lossescited by 114
  28. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  29. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 110
  30. 2022Task-aware Retrieval with Instructionscited by 107
  31. 2026AutoSkill: Experience-Driven Lifelong Learning via Skill Self-Evolutioncited by 105
  32. 2023Lift Yourself Up: Retrieval-augmented Text Generation with Self Memorycited by 103
  33. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  34. 2025On the Theoretical Limitations of Embedding-Based Retrievalcited by 98
  35. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 95
  36. 2022Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval?cited by 90
  37. 2024Is Cosine-Similarity of Embeddings Really About Similarity?cited by 89
  38. 2023Dense X Retrieval: What Retrieval Granularity Should We Use?cited by 88

50 most cited of 1,160 All 1,160 (JSON)