mlsanityData to 7 Oct 2026

A Study on Token Pruning for ColBERT

arXiv 2112.06540 ↗First posted 13 Dec 2021cs.IR cs.CLcited by 14

On the map: Efficient LLM inference and serving → Visual token pruning in multimodal models

Abstract (arXiv)

The ColBERT model has recently been proposed as an effective BERT based ranker. By adopting a late interaction mechanism, a major advantage of ColBERT is that document representations can be precomputed in advance. However, the big downside of the model is the index size, which scales linearly with the number of tokens in the collection. In this paper, we study various designs for ColBERT models in order to attack this problem. While compression techniques have been explored to reduce the index size, in this paper we study token pruning techniques for ColBERT. We compare simple heuristics, as well as a single layer of attention mechanism to select the tokens to keep at indexing time. Our experiments show that ColBERT indexes can be pruned up to 30% on the MS MARCO passage collection without a significant drop in performance. Finally, we experiment on MS MARCO documents, which reveal several challenges for such mechanism.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving. Builds on 16 of its 20 references that are in the map. First posted on arXiv 13 Dec 2021.

Builds on

16 of 20 references.

References · 16

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2020MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformerscited by 1,414
  5. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  6. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  7. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  8. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 266
  9. 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
  10. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
  11. 2021SPLADE v2: Sparse Lexical and Expansion Model for Information Retrievalcited by 126
  12. 2020Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Searchcited by 76
4 more
  1. 2021Efficient Passage Retrieval with Hashing for Open-domain Question Answeringcited by 53
  2. 2021Pseudo-Relevance Feedback for Multiple Representation Dense Retrievalcited by 31
  3. 2021Mitigating the Position Bias of Transformer Models in Passage Re-Rankingcited by 10
  4. 2021Query Embedding Pruning for Dense Retrievalcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 4; 2025: 1; 2026: 5.

14 papers.

Sort:

By area:

  1. 2022Token Merging: Your ViT But Fastercited by 995
  2. 2023Token Fusion: Bridging the Gap between Token Pruning and Token Mergingcited by 123
  3. 2023CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformerscited by 44
  4. 2024Token-level Correlation-guided Compression for Efficient Multimodal Document Understandingcited by 22
  5. 2022Introducing Neural Bag of Whole-Words with ColBERTer: Contextualized Late Interactions using Enhanced Reductioncited by 21
  6. 2024iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Modelscited by 11
  7. 2024PaPr: Training-Free One-Step Patch Pruning with Lightweight ConvNets for Faster Inferencecited by 9
  8. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  9. 2026Multi-Vector Index Compression in Any Modalitycited by 8
  10. 2025LightVLM: Acceleraing Large Multimodal Models with Pyramid Token Merging and KV Cache Compressioncited by 7
  11. 2024ImagePiece: Content-aware Re-tokenization for Efficient Image Recognitioncited by 0
  12. 2026Query-Aware Token Budgeting for Efficient Late-Interaction Visual Document Retrievalcited by 0
2 more
  1. 2026EigenLI: Spectral Approximations to Late Interactioncited by 0
  2. 2026Region-Aware CLS Token Augmentation for Fine-Grained Image Retrievalcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Lossless Token Pruning in Late-Interaction Retrieval Models
  2. 2024Focus on the Core: Efficient Attention via Pruned Token Compression for Document Classification
  3. 2022The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models
  4. 2026CompRank: Efficient LLM Reranking via Token-Level Compression and Decoding-Free Scoring
  5. 2023Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
  6. 2026RenderRank: Learning to Rerank Text with Compressed Visual Tokens
  7. 2025Reranking with Compressed Document Representation ↗
  8. 2025Index-Preserving Lightweight Token Pruning for Efficient Document Understanding in Vision-Language Models