Spatially Grounded Explanations in Vision Language Models for Document Visual Question Answering
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
We introduce EaGERS, a fully training-free and model-agnostic pipeline that (1) generates natural language rationales via a vision language model, (2) grounds these rationales to spatial sub-regions by computing multimodal embedding similarities over a configurable grid with majority voting, and (3) restricts the generation of responses only from the relevant regions selected in the masked image. Experiments on the DocVQA dataset demonstrate that our best configuration not only outperforms the base model on exact match accuracy and Average Normalized Levenshtein Similarity metrics but also enhances transparency and reproducibility in DocVQA without additional model fine-tuning.
Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 21 references that are in the map. First posted on arXiv 15 Jul 2025.
Builds on
20 of 21 references.
Closest ancestors · 1
Same direction, newest first
Other references · 19
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,588
- 2023Visual Instruction Tuningcited by 7,827
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,713
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,103
- 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,855
- 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,362
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,295
- 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,031
- 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 648
- 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 551
- 2021OCR-free Document Understanding Transformercited by 402
- 2019Scene Text Visual Question Answeringcited by 327
7 more
- 2021DocFormer: End-to-End Transformer for Document Understandingcited by 254
- 2021Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformercited by 134
- 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 113
- 2019Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQAcited by 107
- 2024DLaVA: Document Language and Vision Assistant for Answer Localization with Enhanced Interpretability and Trustworthinesscited by 8
- 2020Multimodal grid features and cell pointers for Scene Text Visual Question Answeringcited by 5
- 2024DocXplain: A Novel Model-Agnostic Explainability Method for Document Image Classificationcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ReVA: A Region-Aware Visual Assistant for Visually Grounded Question Answering
- 2026Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering
- 2025Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understanding
- 2026Towards Self-Explainable Document Visual Question Answering with Chain-of-Explanation Predictions
- 2025ProtoVQA: An Adaptable Prototypical Framework for Explainable Fine-Grained Visual Question Answering
- 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities
- 2026VISTAQA: Benchmarking Joint Visual Question Answering and Pixel-Level Evidence
- 2025Exploring Advanced Techniques for Visual Question Answering: A Comprehensive Comparison