mlsanityData to 7 Oct 2026

A Picture is Worth a Thousand Words? An Empirical Study of Aggregation Strategies for Visual Financial Document Retrieval

arXiv 2605.14581 ↗First posted 14 May 2026cs.CV cs.AI cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

Visual RAG has offered an alternative to traditional RAG. It treats documents as images and uses vision encoders to obtain vision patch tokens. However, hundreds of patch tokens per document create retrieval and storage challenges in a vector database. Practical deployment requires aggregating them into a single vector. This raises a critical question: does single-vector aggregation lose key information in financial documents? We develop a diagnostic benchmark using financial documents where changes in single digits can lead to significant semantic shifts. Our experiments show that single-vector aggregation collapses different documents with almost identical vectors. Metrics show that the patch level detects semantic changes, and confirm that aggregation obscures these details. We identify global texture dominance as the root cause. Our findings are consistent across model scales, retrieval-optimized embeddings, and multiple mitigation strategies, highlighting significant risks for single-vector visual document retrieval in financial applications.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 14 of its 20 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

14 of 20 references.

References · 14

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  3. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  4. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 552
  5. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 402
  6. 2026Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Rankingcited by 211
  7. 2024GME: Improving Universal Multimodal Retrieval by Multimodal LLMscited by 166
  8. 2024Unifying Multimodal Retrieval via Document Screenshot Embeddingcited by 107
  9. 2022Towards Complex Document Understanding By Discrete Reasoningcited by 82
  10. 2025TableRAG: A Retrieval Augmented Generation Framework for Heterogeneous Document Reasoningcited by 22
  11. 2025Lost in Embeddings: Information Loss in Vision-Language Modelscited by 18
  12. 2025Efficient Constant-Space Multi-Vector Retrievalcited by 10
2 more
  1. 2025Optimizing Retrieval Strategies for Financial Question Answering Documents in Retrieval-Augmented Generation Systemscited by 9
  2. 2025Metadata-Driven Retrieval-Augmented Generation for Financial Question Answeringcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025FinRAGBench-V: A Benchmark for Multimodal RAG with Visual Citation in the Financial Domain
  2. 2025Rethinking Retrieval: From Traditional Retrieval Augmented Generation to Agentic and Non-Vector Reasoning Systems in the Financial Domain for Large Language Models
  3. 2026PULSAR: Pooled Unified Late-Interaction Search and Retrieval for Enterprise Visual Document RAG
  4. 2025DocPruner: A Storage-Efficient Framework for Multi-Vector Visual Document Retrieval via Adaptive Patch-Level Embedding Pruning
  5. 2026Inverting Multi-Vector Visual Document Indices
  6. 2026From BM25 to Corrective RAG: Benchmarking Retrieval Strategies for Text-and-Table Documents
  7. 2025Reproducibility, Replicability, and Insights into Visual Document Retrieval with Late Interaction
  8. 2025Beyond Patch Aggregation: 3-Pass Pyramid Indexing for Vision-Enhanced Document Retrieval