# A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniques

arXiv: https://arxiv.org/abs/2106.14807 · HTML: https://mlsanity.com/paper/2106.14807/

## Key facts

- Cited by 69 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 25 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 14 of 14)

- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval](https://mlsanity.com/paper/2007.00808/) (2020)
- [RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2010.08191/) (2020)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling](https://mlsanity.com/paper/2104.06967/) (2021)
- [Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation](https://mlsanity.com/paper/2010.02666/) (2020)
- [COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted List](https://mlsanity.com/paper/2104.07186/) (2021)
- [Learning Passage Impacts for Inverted Indexes](https://mlsanity.com/paper/2104.12016/) (2021)
- [Context-Aware Sentence/Passage Term Importance Estimation For First Stage Retrieval](https://mlsanity.com/paper/1910.10687/) (2019)
- [Efficient Passage Retrieval with Hashing for Open-domain Question Answering](https://mlsanity.com/paper/2106.00882/) (2021)
- [A Replication Study of Dense Passage Retriever](https://mlsanity.com/paper/2104.05740/) (2021)
- [Complementing Lexical Retrieval with Semantic Residual Embedding](https://mlsanity.com/paper/2004.13969/) (2020)

## Built on by (most cited first, 30 of 69)

- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://mlsanity.com/paper/2112.01488/) (2021)
- [Query2doc: Query Expansion with Large Language Models](https://mlsanity.com/paper/2303.07678/) (2023)
- [Autoregressive Search Engines: Generating Substrings as Document Identifiers](https://mlsanity.com/paper/2204.10628/) (2022)
- [From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective](https://mlsanity.com/paper/2205.04733/) (2022)
- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?](https://mlsanity.com/paper/2110.06918/) (2021)
- [Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation](https://mlsanity.com/paper/2206.10128/) (2022)
- [Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking](https://mlsanity.com/paper/2310.13243/) (2023)
- [A Survey of Multimodal Retrieval-Augmented Generation](https://mlsanity.com/paper/2504.08748/) (2025)
- [A Survey on Knowledge-Oriented Retrieval-Augmented Generation](https://mlsanity.com/paper/2503.10677/) (2025)
- [A Proposed Conceptual Framework for a Representational Approach to Information Retrieval](https://mlsanity.com/paper/2110.01529/) (2021)
- [Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction](https://mlsanity.com/paper/2203.13088/) (2022)
- [In Defense of Cross-Encoders for Zero-Shot Retrieval](https://mlsanity.com/paper/2212.06121/) (2022)
- [UnifieR: A Unified Retriever for Large-Scale Retrieval](https://mlsanity.com/paper/2205.11194/) (2022)
- [CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval](https://mlsanity.com/paper/2211.10411/) (2022)
- [Planning Ahead in Generative Retrieval: Guiding Autoregressive Generation through Simultaneous Decoding](https://mlsanity.com/paper/2404.14600/) (2024)
- [LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval](https://mlsanity.com/paper/2208.13661/) (2022)
- [Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboard](https://mlsanity.com/paper/2306.07471/) (2023)
- [Multimodal Learned Sparse Retrieval with Probabilistic Expansion Control](https://mlsanity.com/paper/2402.17535/) (2024)
- [RGD: Multi-LLM Based Agent Debugger via Refinement and Generation Guidance](https://mlsanity.com/paper/2410.01242/) (2024)
- [Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answering](https://mlsanity.com/paper/2309.02233/) (2023)
- [Milco: Learned Sparse Retrieval Across Languages via a Multilingual Connector](https://mlsanity.com/paper/2510.00671/) (2025)
- [ESPN: Memory-Efficient Multi-Vector Information Retrieval](https://mlsanity.com/paper/2312.05417/) (2023)
- [Mistral-SPLADE: LLMs for better Learned Sparse Retrieval](https://mlsanity.com/paper/2408.11119/) (2024)
- [Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval](https://mlsanity.com/paper/2506.00041/) (2025)
- [A Survey on Retrieval And Structuring Augmented Generation with Large Language Models](https://mlsanity.com/paper/2509.10697/) (2025)
- [ColBERT-XM: A Modular Multi-Vector Representation Model for Zero-Shot Multilingual Information Retrieval](https://mlsanity.com/paper/2402.15059/) (2024)
- [Exploring the Practicality of Generative Retrieval on Dynamic Corpora](https://mlsanity.com/paper/2305.18952/) (2023)
- [Enhancing Lexicon-Based Text Embeddings with Large Language Models](https://mlsanity.com/paper/2501.09749/) (2025)

## Abstract (arXiv)

Recent developments in representational learning for information retrieval can be organized in a conceptual framework that establishes two pairs of contrasts: sparse vs. dense representations and unsupervised vs. learned representations. Sparse learned representations can further be decomposed into expansion and term weighting components. This framework allows us to understand the relationship between recently proposed techniques such as DPR, ANCE, DeepCT, DeepImpact, and COIL, and furthermore, gaps revealed by our analysis point to "low hanging fruit" in terms of techniques that have yet to be explored. We present a novel technique dubbed "uniCOIL", a simple extension of COIL that achieves to our knowledge the current state-of-the-art in sparse retrieval on the popular MS MARCO passage ranking dataset. Our implementation using the Anserini IR toolkit is built on the Lucene search library and thus fully compatible with standard inverted indexes.