mlsanityData to 14 Oct 2026

Efficient Constant-Space Multi-Vector Retrieval

arXiv 2504.01818 ↗First posted 2 Apr 2025cs.IR cs.CLcited by 12

Not on the map. Why?

Abstract (arXiv)

Multi-vector retrieval methods, exemplified by the ColBERT architecture, have shown substantial promise for retrieval by providing strong trade-offs in terms of retrieval latency and effectiveness. However, they come at a high cost in terms of storage since a (potentially compressed) vector needs to be stored for every token in the input collection. To overcome this issue, we propose encoding documents to a fixed number of vectors, which are no longer necessarily tied to the input tokens. Beyond reducing the storage costs, our approach has the advantage that document representations become of a fixed size on disk, allowing for better OS paging management. Through experiments using the MSMARCO passage corpus and BEIR with the ColBERT-v2 architecture, a representative multi-vector ranking model architecture, we find that passages can be effectively encoded into a fixed number of vectors while retaining most of the original effectiveness.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 23 of its 29 references that are in the map. First posted on arXiv 2 Apr 2025.

Builds on

23 of 29 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2017Billion-scale similarity search with GPUscited by 2,497
  3. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  4. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  5. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  6. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  7. 2020Overview of the TREC 2019 deep learning trackcited by 286
  8. 2020Pretrained Transformers for Text Ranking: BERT and Beyondcited by 267
  9. 2021Overview of the TREC 2020 deep learning trackcited by 196
  10. 2020Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillationcited by 145
  11. 2022RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encodercited by 118
  12. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 101
11 more
  1. 2021Learning Passage Impacts for Inverted Indexescited by 68
  2. 2024MUVERA: Multi-Vector Retrieval via Fixed Dimensional Encodingscited by 32
  3. 2023Rethinking the Role of Token Retrieval in Multi-Vector Retrievalcited by 31
  4. 2022An Efficiency Study for SPLADE Modelscited by 26
  5. 2024Reducing the Footprint of Multi-Vector Retrieval with Minimal Performance Impact via Token Poolingcited by 25
  6. 2020A White Box Analysis of ColBERTcited by 14
  7. 2019Let's measure run time! Extending the IR replicability infrastructure to include performance aspectscited by 10
  8. 2024Faster Learned Sparse Retrieval with Block-Max Pruningcited by 10
  9. 2024A Reproducibility Study of PLAIDcited by 8
  10. 2024DeeperImpact: Optimizing Sparse Learned Index Structurescited by 5
  11. 2021On Single and Multiple Representations in Dense Passage Retrievalcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 8.

12 papers.

  1. 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 20
  2. 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Frameworkcited by 10
  3. 2025Towards Lossless Token Pruning in Late-Interaction Retrieval Modelscited by 2
  4. 2025Simple Projection Variants Improve ColBERT Performancecited by 2
  5. 2025LIR: The First Workshop on Late Interaction and Multi Vector Retrieval @ ECIR 2026cited by 1
  6. 2026LEMUR: Learned Multi-Vector Retrievalcited by 1
  7. 2026AnchorFold: A Focus-Then-Fold Framework via Recursive Attention Propagation for Efficient Multi-Vector Visual Document Retrievalcited by 1
  8. 2026EigenLI: Spectral Approximations to Late Interactioncited by 1
  9. 2026A Picture is Worth a Thousand Words? An Empirical Study of Aggregation Strategies for Visual Financial Document Retrievalcited by 0
  10. 2026H+ Embedding: Harmonizing Global and Token-Level Retrieval with Context-Dependent Phrasescited by 0
  11. 2026Beyond the Final Embedding: Retrieving Memories from Token-Level Evidencecited by 0
  12. 2026Simple Linear Corrections for Compression in Late Interactioncited by 0