mlsanityData to 14 Oct 2026

Visual Late Chunking: An Empirical Study of Contextual Chunking for Efficient Visual Document Retrieval

arXiv 2604.10167 ↗First posted 11 Apr 2026cs.CV cs.CL cs.IRcited by 8

Not on the map. Why?

Abstract (arXiv)

Multi-vector models dominate Visual Document Retrieval (VDR) due to their fine-grained matching capabilities, but their high storage and computational costs present a major barrier to practical deployment. In this paper, we propose ColChunk, a plug-and-play framework that introduces multimodal late chunking to construct efficient, contextualized multi-vectors. Unlike existing pruning or fixed-token approaches, ColChunk employs hierarchical clustering on patch-level embeddings, fused with a 2D position prior to ensure spatial-semantic coherence. This adaptive grouping allows for a content-aware representation that preserves global context while drastically reducing the vector count. Evaluations across 24 VDR datasets demonstrate ColChunk achieves over a 90% reduction in storage requirements while simultaneously delivering a 9-point average improvement in nDCG@5 across representative single-vector models. ColChunk provides a practical solution for balancing retrieval accuracy and efficiency in visual document systems.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 25 of its 43 references that are in the map. First posted on arXiv 11 Apr 2026.

Builds on

25 of 43 references.

References · 25

  1. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
  2. 2024VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documentscited by 245
  3. 2024VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Taskscited by 228
  4. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 170
  5. 2025VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documentscited by 116
  6. 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsingcited by 116
  7. 2024Unifying Multimodal Retrieval via Document Screenshot Embeddingcited by 110
  8. 2025jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrievalcited by 76
  9. 2024Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Modelscited by 73
  10. 2025ViDoRe Benchmark V2: Raising the Bar for Visual Retrievalcited by 61
  11. 2025UniME-V2: MLLM-as-a-Judge for Universal Multimodal Embedding Learningcited by 42
  12. 2025MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interactioncited by 39
13 more
  1. 2023Rethinking the Role of Token Retrieval in Multi-Vector Retrievalcited by 31
  2. 2024Reducing the Footprint of Multi-Vector Retrieval with Minimal Performance Impact via Token Poolingcited by 25
  3. 2025jina-reranker-v3: Last but Not Late Interaction for Listwise Document Rerankingcited by 24
  4. 2025ModernVBERT: Towards Smaller Visual Document Retrieverscited by 21
  5. 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruningcited by 20
  6. 2025Context is Gold to find the Gold Passage: Evaluating and Training Contextual Document Embeddingscited by 13
  7. 2022Multi-Vector Retrieval as Sparse Alignmentcited by 12
  8. 2025Lost in OCR Translation? Vision-Based Approaches to Robust Document Retrievalcited by 11
  9. 2023ESPN: Memory-Efficient Multi-Vector Information Retrievalcited by 8
  10. 2026Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrievalcited by 8
  11. 2026CausalEmbed: Auto-Regressive Multi-Vector Generation in Latent Space for Visual Document Embeddingcited by 7
  12. 2026ReinPool: Reinforcement Learning Pooling Multi-Vector Embeddings for Retrieval Systemcited by 5
  13. 2026Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representationscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Frameworkcited by 2
  2. 2026AnchorFold: A Focus-Then-Fold Framework via Recursive Attention Propagation for Efficient Multi-Vector Visual Document Retrievalcited by 1
  3. 2026EigenLI: Spectral Approximations to Late Interactioncited by 1
  4. 2026VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrievalcited by 0
  5. 2026Spatial Matryoshka Training for Multi-Granularity Visual Document Retrievalcited by 0
  6. 2026MAGIC: Marginal-Guided Compression with Optimal Transport for Efficient Visual Document Retrievalcited by 0
  7. 2026Simple Linear Corrections for Compression in Late Interactioncited by 0
  8. 2026VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrievalcited by 0