# AVIR: Adaptive Visual In-Document Retrieval for Efficient Multi-Page Document Question Answering

arXiv: https://arxiv.org/abs/2601.11976 · HTML: https://mlsanity.com/paper/2601.11976/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 17 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [Scene Text Visual Question Answering](https://mlsanity.com/paper/1905.13648/) (2019)
- [SlideVQA: A Dataset for Document Visual Question Answering on Multiple Images](https://mlsanity.com/paper/2301.04883/) (2023)
- [VisualMRC: Machine Reading Comprehension on Document Images](https://mlsanity.com/paper/2101.11272/) (2021)
- [Document Understanding Dataset and Evaluation (DUDE)](https://mlsanity.com/paper/2305.08455/) (2023)
- [Hierarchical multimodal transformers for Multi-Page DocVQA](https://mlsanity.com/paper/2212.05935/) (2022)
- [DocFormerv2: Local Features for Document Understanding](https://mlsanity.com/paper/2306.01733/) (2023)
- [InstructDoc: A Dataset for Zero-Shot Generalization of Visual Document Understanding with Instructions](https://mlsanity.com/paper/2401.13313/) (2024)
- [FRAG: Frame Selection Augmented Generation for Long Video and Long Document Understanding](https://mlsanity.com/paper/2504.17447/) (2025)
- [GRAM: Global Reasoning for Multi-Page VQA](https://mlsanity.com/paper/2401.03411/) (2024)

## Built on by (most cited first, 4 of 4)

- [BERAG: Bayesian Ensemble Retrieval-Augmented Generation for Knowledge-based Visual Question Answering](https://mlsanity.com/paper/2604.22678/) (2026)
- [Locating Failure in Multi-Page Visually Rich Document Understanding: An Empirical Attribution](https://mlsanity.com/paper/2608.07943/) (2026)
- [What the Reranker Sees: Multi-Aspect Page Annotation for Long-Document Multimodal Question Answering](https://mlsanity.com/paper/2608.14841/) (2026)
- [ViSAR: Training-Free Adaptive-k Retrieval for Visual Document Question Answering](https://mlsanity.com/paper/2609.02486/) (2026)

## Abstract (arXiv)

Multi-page Document Visual Question Answering (MP-DocVQA) remains challenging because long documents not only strain computational resources but also reduce the effectiveness of the attention mechanism in large vision-language models (LVLMs). We tackle these issues with an Adaptive Visual In-document Retrieval (AVIR) framework. A lightweight retrieval model first scores each page for question relevance. Pages are then clustered according to the score distribution to adaptively select relevant content. The clustered pages are screened again by Top-K to keep the context compact. However, for short documents, clustering reliability decreases, so we use a relevance probability threshold to select pages. The selected pages alone are fed to a frozen LVLM for answer generation, eliminating the need for model fine-tuning. The proposed AVIR framework reduces the average page count required for question answering by 70%, while achieving an ANLS of 84.58% on the MP-DocVQA dataset-surpassing previous methods with significantly lower computational cost. The effectiveness of the proposed AVIR is also verified on the SlideVQA and DUDE benchmarks. The code is available at https://github.com/Li-yachuan/AVIR.