# Learning Passage Impacts for Inverted Indexes

arXiv: https://arxiv.org/abs/2104.12016 · HTML: https://mlsanity.com/paper/2104.12016/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2021.

## Built on by (most cited first, 30 of 68)

- [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://mlsanity.com/paper/2112.01488/) (2021)
- [SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval](https://mlsanity.com/paper/2109.10086/) (2021)
- [Autoregressive Search Engines: Generating Substrings as Document Identifiers](https://mlsanity.com/paper/2204.10628/) (2022)
- [PLAID: An Efficient Engine for Late Interaction Retrieval](https://mlsanity.com/paper/2205.09707/) (2022)
- [From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective](https://mlsanity.com/paper/2205.04733/) (2022)
- [A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniques](https://mlsanity.com/paper/2106.14807/) (2021)
- [Salient Phrase Aware Dense Retrieval: Can a Dense Retriever Imitate a Sparse One?](https://mlsanity.com/paper/2110.06918/) (2021)
- [Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation](https://mlsanity.com/paper/2206.10128/) (2022)
- [Scaling Laws For Dense Retrieval](https://mlsanity.com/paper/2403.18684/) (2024)
- [An Efficiency Study for SPLADE Models](https://mlsanity.com/paper/2207.03834/) (2022)
- [A Proposed Conceptual Framework for a Representational Approach to Information Retrieval](https://mlsanity.com/paper/2110.01529/) (2021)
- [Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reduction](https://mlsanity.com/paper/2203.13088/) (2022)
- [In Defense of Cross-Encoders for Zero-Shot Retrieval](https://mlsanity.com/paper/2212.06121/) (2022)
- [Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges](https://mlsanity.com/paper/2403.00784/) (2024)
- [NExtLong: Toward Effective Long-Context Training without Long Documents](https://mlsanity.com/paper/2501.12766/) (2025)
- [CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval](https://mlsanity.com/paper/2211.10411/) (2022)
- [Multiview Identifiers Enhanced Generative Retrieval](https://mlsanity.com/paper/2305.16675/) (2023)
- [Quest: Query-centric Data Synthesis Approach for Long-context Scaling of Large Language Model](https://mlsanity.com/paper/2405.19846/) (2024)
- [LED: Lexicon-Enlightened Dense Retriever for Large-Scale Retrieval](https://mlsanity.com/paper/2208.13661/) (2022)
- [Constructing Tree-based Index for Efficient and Effective Dense Retrieval](https://mlsanity.com/paper/2304.11943/) (2023)
- [Efficient Constant-Space Multi-Vector Retrieval](https://mlsanity.com/paper/2504.01818/) (2025)
- [Augmenting Black-box LLMs with Medical Textbooks for Biomedical Question Answering](https://mlsanity.com/paper/2309.02233/) (2023)
- [CSPLADE: Learned Sparse Retrieval with Causal Language Models](https://mlsanity.com/paper/2504.10816/) (2025)
- [Query-as-context Pre-training for Dense Passage Retrieval](https://mlsanity.com/paper/2212.09598/) (2022)
- [DAPR: A Benchmark on Document-Aware Passage Retrieval](https://mlsanity.com/paper/2305.13915/) (2023)
- [ESPN: Memory-Efficient Multi-Vector Information Retrieval](https://mlsanity.com/paper/2312.05417/) (2023)
- [Mistral-SPLADE: LLMs for better Learned Sparse Retrieval](https://mlsanity.com/paper/2408.11119/) (2024)
- [Decoding Dense Embeddings: Sparse Autoencoders for Interpreting and Discretizing Dense Retrieval](https://mlsanity.com/paper/2506.00041/) (2025)
- [Exploring the Practicality of Generative Retrieval on Dynamic Corpora](https://mlsanity.com/paper/2305.18952/) (2023)
- [Enhancing Lexicon-Based Text Embeddings with Large Language Models](https://mlsanity.com/paper/2501.09749/) (2025)

## Abstract (arXiv)

Neural information retrieval systems typically use a cascading pipeline, in which a first-stage model retrieves a candidate set of documents and one or more subsequent stages re-rank this set using contextualized language models such as BERT. In this paper, we propose DeepImpact, a new document term-weighting scheme suitable for efficient retrieval using a standard inverted index. Compared to existing methods, DeepImpact improves impact-score modeling and tackles the vocabulary-mismatch problem. In particular, DeepImpact leverages DocT5Query to enrich the document collection and, using a contextualized language model, directly estimates the semantic importance of tokens in a document, producing a single-value representation for each token in each document. Our experiments show that DeepImpact significantly outperforms prior first-stage retrieval approaches by up to 17% on effectiveness metrics w.r.t. DocT5Query, and, when deployed in a re-ranking scenario, can reach the same effectiveness of state-of-the-art approaches with up to 5.1x speedup in efficiency.