mlsanityData to 14 Oct 2026

MAGIC: Marginal-Guided Compression with Optimal Transport for Efficient Visual Document Retrieval

arXiv 2609.21018 ↗First posted 17 Sep 2026cs.CV cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent visual document retrieval (VDR) systems such as ColPali use multi-vector page embeddings, in which patch-level vectors enable fine-grained evidence matching but incur substantial index storage and MaxSim scoring overhead. Post-hoc merging offers a practical route to efficient VDR by reducing this cost without retraining the retriever, but its uniform reconstruction objectives are poorly aligned with the sparse, non-uniform patch usage induced by late-interaction retrieval. Under aggressive compression, this misalignment can preserve rarely used patches while concentrating retrieval activity on too few retained representatives. To address this misalignment, we propose Marginal-Guided Compression with Optimal Transport (MAGIC), a training-free post-hoc compressor for efficient retrieval with frozen multi-vector embeddings. MAGIC derives a MaxSim-induced compression surrogate and optimizes it through a two-marginal entropic optimal-transport formulation, where a retrieval-demand source marginal prioritizes high-use patches and a balanced target marginal regularizes retained-facet usage. Across ViDoRe benchmarks, keep ratios, and retrieval backbones, MAGIC consistently outperforms strong post-hoc compressors, with particularly large gains in the aggressive-compression regime; component ablations verify the complementary effects of its two marginals. We release the code at: https://github.com/xandery-geek/MAGIC.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 42 of its 49 references that are in the map. First posted on arXiv 17 Sep 2026.

Builds on

42 of 49 references.

References · 42

  1. 2023Visual Instruction Tuningcited by 7,869
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  3. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  4. 2025Qwen3-VL Technical Reportcited by 2,793
  5. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  7. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  8. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  9. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  10. 2021InfographicVQAcited by 437
  11. 2023Improving Text Embeddings with Large Language Modelscited by 303
  12. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 245
30 more
  1. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 230
  2. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 228
  3. 2024ColPali: Efficient Document Retrieval with Vision Language Modelscited by 195
  4. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 183
  5. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
  6. 2023SlideVQA: A Dataset for Document Visual Question Answering on Multiple Imagescited by 179
  7. 2024MM-Embed: Universal Multimodal Retrieval with Multimodal LLMscited by 161
  8. 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 116
  9. 2024M3DocRAG: Multi-modal Retrieval is What You Need for Multi-page Multi-document Understandingcited by 111
  10. 2024Unifying Multimodal Retrieval via Document Screenshot Embeddingcited by 110
  11. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  12. 2025jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrievalcited by 76
  13. 2025ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic Iterative Reasoning Agentscited by 66
  14. 2025ViDoRe Benchmark V2: Raising the Bar for Visual Retrievalcited by 61
  15. 2025VDocRAG: Retrieval-Augmented Generation over Visually-Rich Documentscited by 59
  16. 2025UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learningcited by 42
  17. 2025MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interactioncited by 39
  18. 2025Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddingscited by 26
  19. 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 20
  20. 2026Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoningcited by 20
  21. 2025HiBench: Benchmarking LLMs Capability on Hierarchical Structure Reasoningcited by 12
  22. 2025mKG-RAG: Leveraging Multimodal Knowledge Graphs in Retrieval-Augmented Generation for Knowledge-intensive VQAcited by 11
  23. 2026Sculpting the Vector Space: Towards Efficient Multi-Vector Visual Document Retrieval via Prune-then-Merge Frameworkcited by 10
  24. 2026Multi-Vector Index Compression in Any Modalitycited by 10
  25. 2024Instruction-guided Multi-Granularity Segmentation and Captioning with Large Multimodal Modelcited by 8
  26. 2026Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrievalcited by 8
  27. 2026SUPERGLASSES: Benchmarking Vision Language Models as Intelligent Agents for AI Smart Glassescited by 6
  28. 2023Semantic-Aware Adversarial Training for Reliable Deep Hashing Retrievalcited by 5
  29. 2026COT-FM: Cluster-wise Optimal Transport Flow Matchingcited by 3
  30. 2026A Voronoi Cell Formulation for Principled Token Pruning in Late-Interaction Retrieval Modelscited by 2

Built on by

Not cited yet.