mlsanityData to 7 Oct 2026

COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List

arXiv 2104.07186 ↗First posted 15 Apr 2021cs.IR cs.CLcited by 113

Not on the map. Why?

Abstract (arXiv)

Classical information retrieval systems such as BM25 rely on exact lexical match and carry out search efficiently with inverted list index. Recent neural IR models shifts towards soft semantic matching all query document terms, but they lose the computation efficiency of exact match systems. This paper presents COIL, a contextualized exact match retrieval architecture that brings semantic lexical matching. COIL scoring is based on overlapping query document tokens' contextualized representations. The new architecture stores contextualized token representations in inverted lists, bringing together the efficiency of exact match and the representation power of deep language models. Our experimental results show COIL outperforms classical lexical retrievers and state-of-the-art deep LM retrievers with similar or smaller latency.

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). Builds on 24 of its 39 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

24 of 39 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  7. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  8. 2017Billion-scale similarity search with GPUscited by 2,477
  9. 2018Deep contextualized word representationscited by 2,285
  10. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  11. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  12. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,124
12 more
  1. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  2. 2019Passage Re-ranking with BERTcited by 629
  3. 2020Overview of the TREC 2019 deep learning trackcited by 282
  4. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 213
  5. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 164
  6. 2017A Deep Relevance Matching Model for Ad-hoc Retrievalcited by 126
  7. 2019Deeper Text Understanding for IR with Contextual Neural Language Modelingcited by 111
  8. 2017End-to-End Neural Ad-hoc Ranking with Kernel Poolingcited by 85
  9. 2019Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrievalcited by 65
  10. 2021Rethink Training of BERT Rerankers in Multi-Stage Retrieval Pipelinecited by 65
  11. 2020Efficient Document Re-Ranking for Transformers by Precomputing Term Representationscited by 44
  12. 2016Query Expansion with Locally-Trained Word Embeddingscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 41; 2023: 21; 2024: 11; 2025: 16; 2026: 13.

113 papers.

Sort:

By area:

  1. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  2. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  3. 2022Promptagator: Few-shot Dense Retrieval From 8 Examplescited by 233
  4. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  5. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
  6. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  7. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
  8. 2022Autoregressive Search Engines: Generating Substrings as Document Identifierscited by 112
  9. 2022From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effectivecited by 98
  10. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 95
  11. 2021Adversarial Retriever-Ranker for dense text retrievalcited by 82
  12. 2023Llama2Vec: Unsupervised Adaptation of Large Language Models for Dense Retrievalcited by 77
38 more
  1. 2021A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniquescited by 69
  2. 2021Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?cited by 59
  3. 2024CoIR: A Comprehensive Benchmark for Code Information Retrieval Modelscited by 59
  4. 2022Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generationcited by 47
  5. 2022Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformercited by 45
  6. 2022Multi-View Document Representation Learning for Open-Domain Dense Retrievalcited by 41
  7. 2022ERNIE-Search: Bridging Cross-Encoder with Dual-Encoder via Self On-the-fly Distillation for Dense Passage Retrievalcited by 41
  8. 2022Towards Robust Ranker for Text Retrievalcited by 37
  9. 2022Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Modelscited by 37
  10. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  11. 2022OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answeringcited by 29
  12. 2023Rethinking the Role of Token Retrieval in Multi-Vector Retrievalcited by 28
  13. 2022Multi-CPR: A Multi Domain Chinese Dataset for Passage Retrievalcited by 26
  14. 2022An Efficiency Study for SPLADE Modelscited by 26
  15. 2021A Proposed Conceptual Framework for a Representational Approach to Information Retrievalcited by 24
  16. 2022LaPraDoR: Unsupervised Pretrained Dense Retriever for Zero-Shot Text Retrievalcited by 24
  17. 2022Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reductioncited by 21
  18. 2022SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrievalcited by 21
  19. 2022UnifieR: A Unified Retriever for Large-Scale Retrievalcited by 19
  20. 2022CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrievalcited by 19
  21. 2022You can't pick your neighbors, or can you? When and how to rely on retrieval in the kNN-LMcited by 18
  22. 2023Large Search Model: Redefining Search Stack in the Era of LLMscited by 18
  23. 2022Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrievalcited by 17
  24. 2022No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrievalcited by 17
  25. 2022ConTextual Masked Auto-Encoder for Dense Passage Retrievalcited by 16
  26. 2022DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Enginecited by 15
  27. 2022LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrievalcited by 15
  28. 2023Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboardcited by 15
  29. 2024Ask-EDA: A Design Assistant Empowered by LLM, Hybrid RAG and Abbreviation De-hallucinationcited by 14
  30. 2022Multi-Vector Retrieval as Sparse Alignmentcited by 12
  31. 2023BERM: Training the Balanced and Extractable Representation for Matching to Improve Generalization Ability of Dense Retrievalcited by 11
  32. 2023Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answeringcited by 11
  33. 2022Incorporating Explicit Knowledge in Pre-trained Language Models for Passage Re-rankingcited by 10
  34. 2021TLDR: Twin Learning for Dimensionality Reductioncited by 9
  35. 2022ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inferencecited by 9
  36. 2022Query-as-context Pre-training for Dense Passage Retrievalcited by 9
  37. 2022Isotropic Representation Can Improve Dense Retrievalcited by 8
  38. 2024Generative Retrieval as Multi-Vector Dense Retrievalcited by 8

50 most cited of 113 All 113 (JSON)