# DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruning

arXiv: https://arxiv.org/abs/2509.23883 · HTML: https://mlsanity.com/paper/2509.23883/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 96 of its 110 references that are in the map.
- First posted on arXiv 28 Sep 2025.

## Builds on (most cited first, 30 of 96)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation](https://mlsanity.com/paper/2402.03216/) (2024)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [PaliGemma: A versatile 3B VLM for transfer](https://mlsanity.com/paper/2407.07726/) (2024)
- [An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models](https://mlsanity.com/paper/2403.06764/) (2024)
- [LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models](https://mlsanity.com/paper/2407.07895/) (2024)
- [C-Pack: Packed Resources For General Chinese Embeddings](https://mlsanity.com/paper/2309.07597/) (2023)
- [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://mlsanity.com/paper/2112.01488/) (2021)
- [SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference](https://mlsanity.com/paper/2410.04417/) (2024)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023)
- [VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks](https://mlsanity.com/paper/2410.05160/) (2024)
- [mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding](https://mlsanity.com/paper/2403.12895/) (2024)
- [Whitening Sentence Representations for Better Semantics and Faster Retrieval](https://mlsanity.com/paper/2103.15316/) (2021)
- [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023)
- [Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers](https://mlsanity.com/paper/2506.23918/) (2025)
- [ColPali: Efficient Document Retrieval with Vision Language Models](https://mlsanity.com/paper/2407.01449/) (2024)
- [Agent AI: Surveying the Horizons of Multimodal Interaction](https://mlsanity.com/paper/2401.03568/) (2024)
- [GME: Improving Universal Multimodal Retrieval by Multimodal LLMs](https://mlsanity.com/paper/2412.16855/) (2024)
- [MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs](https://mlsanity.com/paper/2411.02571/) (2024)
- [Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs](https://mlsanity.com/paper/2412.01818/) (2024)
- [The Revolution of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2402.12451/) (2024)
- [Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning](https://mlsanity.com/paper/2401.06805/) (2024)
- [Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Models](https://mlsanity.com/paper/2409.10197/) (2024)

## Built on by (most cited first, 20 of 20)

- [Don't Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context Retention](https://mlsanity.com/paper/2510.02912/) (2025)
- [Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Framework](https://mlsanity.com/paper/2602.19549/) (2026)
- [Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding](https://mlsanity.com/paper/2510.15253/) (2025)
- [Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval](https://mlsanity.com/paper/2602.19961/) (2026)
- [Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrieval](https://mlsanity.com/paper/2604.10167/) (2026)
- [CausalEmbed: Auto-Regressive Multi-Vector Generation in Latent Space for Visual Document Embedding](https://mlsanity.com/paper/2601.21262/) (2026)
- [Structural Anchor Pruning: Training-Free Multi-Vector Compression for Visual Document Retrieval](https://mlsanity.com/paper/2601.20107/) (2026)
- [Hybrid-Vector Retrieval for Visually Rich Documents: Combining Single-Vector Efficiency and Multi-Vector Accuracy](https://mlsanity.com/paper/2510.22215/) (2025)
- [Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representations](https://mlsanity.com/paper/2603.01666/) (2026)
- [Roles of MLLMs in Visually Rich Document Retrieval for RAG: A Survey](https://mlsanity.com/paper/2601.03262/) (2025)
- [NanoVDR: Distilling a 2B Vision-Language Retriever into a 70M Text-Only Encoder for Visual Document Retrieval](https://mlsanity.com/paper/2603.12824/) (2026)
- [MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework](https://mlsanity.com/paper/2606.07654/) (2026)
- [LEMUR: Learned Multi-Vector Retrieval](https://mlsanity.com/paper/2601.21853/) (2026)
- [AnchorFold: A Focus-Then-Fold Framework via Recursive Attention Propagation for Efficient Multi-Vector Visual Document Retrieval](https://mlsanity.com/paper/2608.08732/) (2026)
- [EigenLI: Spectral Approximations to Late Interaction](https://mlsanity.com/paper/2609.07561/) (2026)
- [Spatial Matryoshka Training for Multi-Granularity Visual Document Retrieval](https://mlsanity.com/paper/2608.29951/) (2026)
- [Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation](https://mlsanity.com/paper/2608.30163/) (2026)
- [MAGIC: Marginal-Guided Compression with Optimal Transport for Efficient Visual Document Retrieval](https://mlsanity.com/paper/2609.21018/) (2026)
- [AdaMerge: Tuning-Free Patch Compression for Multi-Vector Visual Document Retrieval](https://mlsanity.com/paper/2609.22562/) (2026)
- [Simple Linear Corrections for Compression in Late Interaction](https://mlsanity.com/paper/2610.14196/) (2026)

## Abstract (arXiv)

Visual Document Retrieval (VDR), the task of retrieving visually-rich document pages using queries that combine visual and textual cues, is crucial for numerous real-world applications. Recent state-of-the-art methods leverage Large Vision-Language Models (LVLMs) in a multi-vector paradigm, representing each document as patch-level embeddings to capture fine-grained details. While highly effective, this approach introduces a critical challenge: prohibitive storage overhead, as storing hundreds of vectors per page makes large-scale deployment costly and impractical. To address this, we introduce DocPruner, the first framework to employ adaptive patch-level embedding pruning for VDR to effectively reduce the storage overhead. DocPruner leverages the intra-document patch attention distribution to dynamically identify and discard redundant embeddings for each document. This adaptive mechanism enables a significant 50-60% reduction in storage for leading multi-vector VDR models with negligible degradation in document retrieval performance. Extensive experiments across more than ten representative datasets validate that DocPruner offers a robust, flexible, and effective solution for building storage-efficient, large-scale VDR systems.