# Document understanding and multimodal RAG

Area: [Multimodal LLMs / vision-language models](https://mlsanity.com/mllm/index.md). HTML: https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/

## Key facts

- Document understanding and multimodal RAG: 480 papers on arXiv since 2021 (data through 14 Oct 2026).
- 27 of its 480 mapped papers (6%) cite “TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document” (2024, arXiv 2403.04473).
- 20 of its 480 mapped papers (4%) cite “OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models” (2023, arXiv 2305.07895).
- 21 new papers 17 Sep–14 Oct.

## Most cited here

- [TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document](https://mlsanity.com/paper/2403.04473/) (2024), cited by 27 papers here
- [OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models](https://mlsanity.com/paper/2305.07895/) (2023), cited by 20 papers here
- [ColPali: Efficient Document Retrieval with Vision Language Models](https://mlsanity.com/paper/2407.01449/) (2024), cited by 17 papers here
- [OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning](https://mlsanity.com/paper/2501.00321/) (2024), cited by 17 papers here
- [mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding](https://mlsanity.com/paper/2307.02499/) (2023), cited by 16 papers here
- [E5-V: Universal Embeddings with Multimodal Large Language Models](https://mlsanity.com/paper/2407.12580/) (2024), cited by 13 papers here
- [VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks](https://mlsanity.com/paper/2410.05160/) (2024), cited by 13 papers here
- [MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs](https://mlsanity.com/paper/2411.02571/) (2024), cited by 13 papers here
- [olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models](https://mlsanity.com/paper/2502.18443/) (2025), cited by 12 papers here
- [Wiki-LLaVA: Hierarchical Retrieval-Augmented Generation for Multimodal LLMs](https://mlsanity.com/paper/2404.15406/) (2024), cited by 11 papers here

## New (17 Sep–14 Oct)

- [What Breaks in Multi-Image Schema Application? Factorial Oracle Interventions Reveal an Instance-Information Bottleneck](https://mlsanity.com/paper/2610.19611/) (2026-10-14)
- [Heading Hierarchy Reconstruction for Complex and Long Documents with Level-Wise Multimodal Memory](https://mlsanity.com/paper/2610.18238/) (2026-10-13)
- [From Pixels to Structure: Lightweight Vision-Language Models for Document OCR and Structured JSON Extraction](https://mlsanity.com/paper/2610.11818/) (2026-10-08)
- [Multimodal Graph Retrieval-Augmented Sequential Recommendation via Collaborative Filtering Paths](https://mlsanity.com/paper/2610.11228/) (2026-10-08)
- [SP-DocReader: Difference-Aware Self-Play for Precise Document OCR](https://mlsanity.com/paper/2610.11148/) (2026-10-08)
- [OpticalRec: Unified Optical Vision-Language Representation for Multimodal Recommendation](https://mlsanity.com/paper/2610.05432/) (2026-10-04)
- [CLIMB: Confidence-Guided Complementary Evidence for Multimodal Retrieval-Augmented Generation](https://mlsanity.com/paper/2610.03421/) (2026-10-02)
- [Found but Not Read: When Extracted Text Closes the Retrieval-Reading Gap in Document Vision-Language Models](https://mlsanity.com/paper/2610.02880/) (2026-10-02)
- [AMBER: Multi-View Adaptive Budget Allocation for Listwise Vision-Language Reranking](https://mlsanity.com/paper/2610.02831/) (2026-10-02)
- [Walking the Embedding Space: Datastore Extraction from Multimodal RAG](https://mlsanity.com/paper/2610.01871/) (2026-10-01)
- [Spherical Interpolation for Backward-Compatible Multimodal Representations](https://mlsanity.com/paper/2609.39836/) (2026-09-30)
- [ResComEmb: Effective and Efficient Multimodal Embedding via Residual Homogeneity Compression](https://mlsanity.com/paper/2609.37225/) (2026-09-29)
- [Xiaomi-OCR-0 Technical Report](https://mlsanity.com/paper/2609.36136/) (2026-09-28)
- [RenderRank: Learning to Rerank Text with Compressed Visual Tokens](https://mlsanity.com/paper/2609.35069/) (2026-09-28)
- [ColNanoVDR: Document-Free Query Distillation for Multi-Vector Visual Document Retrieval via Optimal Transport](https://mlsanity.com/paper/2609.34899/) (2026-09-28)
- [RidgeRank: Efficient Visual Document Reranking via Score Fusion and a Shallow Linear Readout](https://mlsanity.com/paper/2609.34192/) (2026-09-28)
- [Learning Multimodal Embeddings with Evidence-Aligned Readout](https://mlsanity.com/paper/2609.33659/) (2026-09-27)
- [Concepts Complement Dense Semantics: Learning Compact Sparse Spaces for Text-Image Retrieval](https://mlsanity.com/paper/2609.32671/) (2026-09-26)
- [Sorry Robot, Happy Human: Vision-Language Models Read Only One of Two Legible Typographic Layers](https://mlsanity.com/paper/2609.31403/) (2026-09-25)
- [An Empirical Study of VLM Pipelines for Long-Document QA](https://mlsanity.com/paper/2609.29933/) (2026-09-24)
- [A Free Lunch? Adapting PP-OCRv6 for Historical Text Recognition](https://mlsanity.com/paper/2609.20064/) (2026-09-17)