# Expansion via Prediction of Importance with Contextualization

arXiv: https://arxiv.org/abs/2004.14245 · HTML: https://mlsanity.com/paper/2004.14245/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Apr 2020.

## Built on by (most cited first, 26 of 26)

- [SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval](https://mlsanity.com/paper/2109.10086/) (2021)
- [mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset](https://mlsanity.com/paper/2108.13897/) (2021)
- [What Are You Token About? Dense Retrieval as Distributions Over the Vocabulary](https://mlsanity.com/paper/2212.10380/) (2022)
- [Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges](https://mlsanity.com/paper/2403.00784/) (2024)
- [VisualSparta: An Embarrassingly Simple Approach to Large-scale Text-to-Image Search with Weighted Bag-of-words](https://mlsanity.com/paper/2101.00265/) (2021)
- [Multimodal Learned Sparse Retrieval with Probabilistic Expansion Control](https://mlsanity.com/paper/2402.17535/) (2024)
- [Evaluating Generative Ad Hoc Information Retrieval](https://mlsanity.com/paper/2311.04694/) (2023)
- [CSPLADE: Learned Sparse Retrieval with Causal Language Models](https://mlsanity.com/paper/2504.10816/) (2025)
- [Milco: Learned Sparse Retrieval Across Languages via a Multilingual Connector](https://mlsanity.com/paper/2510.00671/) (2025)
- [Towards Competitive Search Relevance For Inference-Free Learned Sparse Retrievers](https://mlsanity.com/paper/2411.04403/) (2024)
- [Lexically-Accelerated Dense Retrieval](https://mlsanity.com/paper/2307.16779/) (2023)
- [Shallow Cross-Encoders for Low-Latency Retrieval](https://mlsanity.com/paper/2403.20222/) (2024)
- [Few-shot Prompting for Pairwise Ranking: An Effective Non-Parametric Retrieval Model](https://mlsanity.com/paper/2409.17745/) (2024)
- [Exploring Classic and Neural Lexical Translation Models for Information Retrieval: Interpretability, Effectiveness, and Efficiency Benefits](https://mlsanity.com/paper/2102.06815/) (2021)
- [On Single and Multiple Representations in Dense Passage Retrieval](https://mlsanity.com/paper/2108.06279/) (2021)
- [Adapting Learned Sparse Retrieval for Long Documents](https://mlsanity.com/paper/2305.18494/) (2023)
- [An Alternative to FLOPS Regularization to Effectively Productionize SPLADE-Doc](https://mlsanity.com/paper/2505.15070/) (2025)
- [CodaRAG: Connecting the Dots with Associativity Inspired by Complementary Learning](https://mlsanity.com/paper/2604.10426/) (2026)
- [Rewrite-to-Rank: Optimizing Ad Visibility via Retrieval-Aware Text Rewriting](https://mlsanity.com/paper/2507.21099/) (2025)
- [Improving Transformer-Kernel Ranking Model Using Conformer and Query Term Independence](https://mlsanity.com/paper/2104.09393/) (2021)
- [Exploring ℓ0 Sparsification for Inference-free Sparse Retrievers](https://mlsanity.com/paper/2504.14839/) (2025)
- [Efficient Sketching and Nearest Neighbor Search Algorithms for Sparse Vector Sets](https://mlsanity.com/paper/2509.24815/) (2025)
- [Explainability of Text Processing and Retrieval Methods: A Survey](https://mlsanity.com/paper/2212.07126/) (2022)
- [ERU-KG: Efficient Reference-aligned Unsupervised Keyphrase Generation](https://mlsanity.com/paper/2505.24219/) (2025)
- [To Case or Not to Case: An Empirical Study in Learned Sparse Retrieval](https://mlsanity.com/paper/2601.17500/) (2026)
- [From Tokens to Concepts: Leveraging SAE for SPLADE](https://mlsanity.com/paper/2604.21511/) (2026)

## Abstract (arXiv)

The identification of relevance with little textual context is a primary challenge in passage retrieval. We address this problem with a representation-based ranking approach that: (1) explicitly models the importance of each term using a contextualized language model; (2) performs passage expansion by propagating the importance to similar terms; and (3) grounds the representations in the lexicon, making them interpretable. Passage representations can be pre-computed at index time to reduce query-time latency. We call our approach EPIC (Expansion via Prediction of Importance with Contextualization). We show that EPIC significantly outperforms prior importance-modeling and document expansion approaches. We also observe that the performance is additive with the current leading first-stage retrieval methods, further narrowing the gap between inexpensive and cost-prohibitive passage ranking approaches. Specifically, EPIC achieves a MRR@10 of 0.304 on the MS-MARCO passage ranking dataset with 78ms average query latency on commodity hardware. We also find that the latency is further reduced to 68ms by pruning document representations, with virtually no difference in effectiveness.