# ReAlign: Optimizing the Visual Document Retriever with Reasoning-Guided Fine-Grained Alignment

arXiv: https://arxiv.org/abs/2604.07419 · HTML: https://mlsanity.com/paper/2604.07419/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 22 of its 96 references that are in the map.
- First posted on arXiv 8 Apr 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 22 of 22)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models](https://mlsanity.com/paper/2508.06471/) (2025)
- [Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning](https://mlsanity.com/paper/2505.15966/) (2025)
- [TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document](https://mlsanity.com/paper/2403.04473/) (2024)
- [PaddleOCR 3.0 Technical Report](https://mlsanity.com/paper/2507.05595/) (2025)
- [E5-V: Universal Embeddings with Multimodal Large Language Models](https://mlsanity.com/paper/2407.12580/) (2024)
- [Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs](https://mlsanity.com/paper/2505.15436/) (2025)
- [VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning](https://mlsanity.com/paper/2505.22019/) (2025)
- [ViDoRe Benchmark V2: Raising the Bar for Visual Retrieval](https://mlsanity.com/paper/2505.17166/) (2025)
- [TextHawk: Exploring Efficient Fine-Grained Perception of Multimodal Large Language Models](https://mlsanity.com/paper/2404.09204/) (2024)
- [ModernVBERT: Towards Smaller Visual Document Retrievers](https://mlsanity.com/paper/2510.01149/) (2025)
- [Towards Visual Text Grounding of Multimodal Large Language Model](https://mlsanity.com/paper/2504.04974/) (2025)
- [RegionRAG: Region-level Retrieval-Augmented Generation for Visual Document Understanding](https://mlsanity.com/paper/2510.27261/) (2025)
- [HKRAG: Holistic Knowledge Retrieval-Augmented Generation Over Visually-Rich Documents](https://mlsanity.com/paper/2511.20227/) (2025)
- [Multimodal Reference Visual Grounding](https://mlsanity.com/paper/2504.02876/) (2025)
- [Look as You Think: Unifying Reasoning and Visual Evidence Attribution for Verifiable Document RAG via Reinforcement Learning](https://mlsanity.com/paper/2511.12003/) (2025)
- [Attention Grounded Enhancement for Visual Document Retrieval](https://mlsanity.com/paper/2511.13415/) (2025)

## Built on by (most cited first, 3 of 3)

- [CC-OCR V2: Fine-Grained Attribution of LMM Failures in Real-World Visual Document Understanding](https://mlsanity.com/paper/2605.03903/) (2026)
- [ConceptFormer: Learning Adaptive Latent Concepts for Query-Document Alignment in Visual Document Retrieval](https://mlsanity.com/paper/2608.15698/) (2026)
- [VisDocAgentBench: Benchmarking Agents for Visually Rich Document Retrieval](https://mlsanity.com/paper/2608.17889/) (2026)

## Abstract (arXiv)

Visual document retrieval aims to retrieve a set of document pages relevant to a query from visually rich collections. Existing methods often employ Vision-Language Models (VLMs) to encode queries and visual pages into a shared embedding space, which is then optimized via contrastive training. However, during visual document representation, localized evidence is usually scattered across complex document layouts, making it difficult for retrieval models to capture crucial cues for effective embedding learning. In this paper, we propose Reasoning-Guided Alignment (ReAlign), a method that enhances visual document retrieval by leveraging the reasoning capability of VLMs to provide fine-grained visual document descriptions as supervision signals for training. Specifically, ReAlign employs a superior VLM to identify query-related regions on a page and then generates a query-aware description grounding the cropped visual regions. The retriever is then trained using these region-focused descriptions to align the semantics between queries and visual documents by encouraging the document ranking distribution induced by the region-focused descriptions to match that induced by the original query. Experiments on diverse visually rich document retrieval benchmarks demonstrate that ReAlign consistently improves visual document retrieval performance on both in-domain and out-of-domain datasets, achieving up to 2% relative improvements. Moreover, the advantages of ReAlign generalize across different VLM backbones by guiding models to better focus their attention on critical visual cues for document representation. All code and datasets are available at https://github.com/NEUIR/ReAlign.