mlsanityData to 7 Oct 2026

Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework

arXiv 2602.19549 ↗First posted 23 Feb 2026cs.CL cs.CV cs.IRcited by 8

Not on the map. Why?

Abstract (arXiv)

Visual Document Retrieval (VDR), which aims to retrieve relevant pages within vast corpora of visually-rich documents, is of significance in current multimodal retrieval applications. The state-of-the-art multi-vector paradigm excels in performance but suffers from prohibitive overhead, a problem that current efficiency methods like pruning and merging address imperfectly, creating a difficult trade-off between compression rate and feature fidelity. To overcome this dilemma, we introduce Prune-then-Merge, a novel two-stage framework that synergizes these complementary approaches. Our method first employs an adaptive pruning stage to filter out low-information patches, creating a refined, high-signal set of embeddings. Subsequently, a hierarchical merging stage compresses this pre-filtered set, effectively summarizing semantic content without the noise-induced feature dilution seen in single-stage methods. Extensive experiments on 29 VDR datasets demonstrate that our framework consistently outperforms existing methods, significantly extending the near-lossless compression range and providing robust performance at high compression ratios.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 72 of its 106 references that are in the map. First posted on arXiv 23 Feb 2026.

Builds on

72 of 106 references.

References · 72

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  5. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  6. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  7. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  8. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  9. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
  10. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 556
  11. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
  12. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
38 more
  1. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 222
  2. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 175
  3. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
  4. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 166
  5. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 159
  6. 2024Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMscited by 153
  7. 2024Unifying Multimodal Retrieval via Document Screenshot Embeddingcited by 107
  8. 2024LamRA: Large Multimodal Model as Your Advanced Retrieval Assistantcited by 106
  9. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 95
  10. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 84
  11. 2024MegaPairs: Massive Data Synthesis For Universal Multimodal Retrievalcited by 84
  12. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  13. 2023UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Webcited by 77
  14. 2025Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?cited by 76
  15. 2025Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMscited by 73
  16. 2025jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrievalcited by 72
  17. 2025ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agentscited by 64
  18. 2025ViDoRe Benchmark V2: Raising the Bar for Visual Retrievalcited by 58
  19. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  20. 2025UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learningcited by 41
  21. 2025UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddingscited by 37
  22. 2025MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interactioncited by 36
  23. 2025REAL-MM-RAG: A Real-World Multi-Modal Retrieval Benchmarkcited by 34
  24. 2025Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoningcited by 33
  25. 2025Retrieval Augmented Generation and Understanding in Vision: A Survey and New Outlookcited by 32
  26. 2025CAFe: Unifying Representation and Generation with Contrastive-Autoregressive Finetuningcited by 31
  27. 2025MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Modelscited by 29
  28. 2025RzenEmbed: Towards Comprehensive Multimodal Retrievalcited by 28
  29. 2025Breaking the Batch Barrier (B3) of Contrastive Learning via Smart Batch Miningcited by 27
  30. 2025Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddingscited by 26
  31. 2024Jina-ColBERT-v2: A General-Purpose Multilingual Late Interaction Retrievercited by 25
  32. 2025MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detectioncited by 24
  33. 2025Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Modelcited by 24
  34. 2024Reducing the Footprint of Multi-Vector Retrieval with Minimal Performance Impact via Token Poolingcited by 23
  35. 2025Improve Multi-Modal Embedding Learning via Explicit Hard Negative Gradient Amplifyingcited by 21
  36. 2022CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrievalcited by 19
  37. 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 19
  38. 2025ModernVBERT: Towards Smaller Visual Document Retrieverscited by 19

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representationscited by 3
  2. 2026MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Frameworkcited by 2
  3. 2026AnchorFold: A Focus-Then-Fold Framework via Recursive Attention Propagation for Efficient Multi-Vector Visual Document Retrievalcited by 1
  4. 2026DistilVDR: A Compact End-to-End Visual Document Retriever via Dual-Student Distillationcited by 0
  5. 2026VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrievalcited by 0
  6. 2026Spatial Matryoshka Training for Multi-Granularity Visual Document Retrievalcited by 0
  7. 2026EigenLI: Spectral Approximations to Late Interactioncited by 0
  8. 2026MAGIC: Marginal-Guided Compression with Optimal Transport for Efficient Visual Document Retrievalcited by 0