mlsanityData to 7 Oct 2026

Towards Lossless Token Pruning in Late-Interaction Retrieval Models

arXiv 2504.12778 ↗First posted 17 Apr 2025cs.IR cs.AI cs.CLcited by 2

On the map: Efficient LLM inference and serving → Visual token pruning in multimodal models

Abstract (arXiv)

Late interaction neural IR models like ColBERT offer a competitive effectiveness-efficiency trade-off across many benchmarks. However, they require a huge memory space to store the contextual representation for all the document tokens. Some works have proposed using either heuristics or statistical-based techniques to prune tokens from each document. This however doesn't guarantee that the removed tokens have no impact on the retrieval score. Our work uses a principled approach to define how to prune tokens without impacting the score between a document and a query. We introduce three regularization losses, that induce a solution with high pruning ratios, as well as two pruning strategies. We study them experimentally (in and out-domain), showing that we can preserve ColBERT's performance while using only 30% of the tokens.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving. Builds on 27 of its 50 references that are in the map. First posted on arXiv 17 Apr 2025.

Builds on

27 of 50 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2017Billion-scale similarity search with GPUscited by 2,477
  6. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  7. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  8. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  9. 2019Passage Re-ranking with BERTcited by 629
  10. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  11. 2020Sparse, Dense, and Attentional Representations for Text Retrievalcited by 213
  12. 2021RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-rankingcited by 175
15 more
  1. 2021Condenser: a Pre-training Architecture for Dense Retrievalcited by 162
  2. 2021COIL: Revisit Exact Lexical Match in Information Retrieval with Contextualized Inverted Listcited by 113
  3. 2018Solving Linear Programs in the Current Matrix Multiplication Timecited by 100
  4. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 95
  5. 2017End-to-End Neural Ad-hoc Ranking with Kernel Poolingcited by 85
  6. 2024SPLADE-v3: New baselines for SPLADEcited by 69
  7. 2024Anisotropy Is Inherent to Self-Attention in Transformerscited by 62
  8. 2019Solving Empirical Risk Minimization in the Current Matrix Multiplication Timecited by 53
  9. 2022Multi-View Document Representation Learning for Open-Domain Dense Retrievalcited by 41
  10. 2024MUVERA: Multi-Vector Retrieval via Fixed Dimensional Encodingscited by 30
  11. 2022Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reductioncited by 21
  12. 2022CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrievalcited by 19
  13. 2023SLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexescited by 12
  14. 2025Efficient Constant-Space Multi-Vector Retrievalcited by 10
  15. 2024An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Modelscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Multi-Vector Index Compression in Any Modalitycited by 8
  2. 2026AdaMerge: Tuning-Free Patch Compression for Multi-Vector Visual Document Retrievalcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021A Study on Token Pruning for ColBERT
  2. 2026FLASH-MAXSIM: IO-Aware Fused Kernels for Late-Interaction Retrieval ↗
  3. 2026EntroPrefill: Renyi-Guided Context Pruning with Conditional Stability Guarantees for Retrieval-Augmented Generation
  4. 2023Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrieval
  5. 2024Focus on the Core: Efficient Attention via Pruned Token Compression for Document Classification
  6. 2026Latent Abstraction for Retrieval-Augmented Generation
  7. 2025Provence: efficient and robust context pruning for retrieval-augmented generation
  8. 2024SpeechPrune: Context-aware Token Pruning for Speech Information Retrieval