Magnifying What Matters: Attention-Guided Adaptive Rendering for Visual Text Comprehension
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
Visual Text Comprehension (VTC) renders text into images for a vision-language model (VLM) to read, sidestepping LLM context-window limits and powering applications from long-page OCR to multi-page memory QA. Yet existing VTC pipelines treat rendering and layout as a fixed, content-agnostic preprocessing step and offer little mechanistic understanding of how VLMs internally process visualized text. Through a focused empirical study on VTC QA tasks, we reveal that VLMs exhibit a localization-without-utilization regime: evidence-localizing attention emerges sharply in the middle-to-late layers and is largely decoupled from answer correctness, yet simply enlarging the localized spans on the rendered page recovers a large fraction of the failures. Building on these observations, we propose AGAR (Attention-Guided Adaptive Rendering), a training-free, model-agnostic method that leverages a VLM's own middle-to-late layer attention to identify the top-K important visual patches, maps them back to word spans, and re-renders the page with those spans enlarged before re-inferring the answer. Extensive experiments across nine VTC benchmarks (short-form, long-context, and multi-page memory QA) and four VLM backbones show that AGAR (i)consistently improves off-the-shelf VLMs as a plug-and-play enhancement, (ii)composes with VLM post-training to yield further gains, and (iii)remains robust under both visual- and text-side input degradation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 37 references that are in the map. First posted on arXiv 11 Jun 2026.
Builds on
30 of 37 references.
Closest ancestors · 2
Same direction, newest first
- 2026VISTA-Bench: Do Vision-Language Models Really Understand Visualized Text as Well as Pure Text?4 Feb
- 2025Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs20 Oct
Other references · 28
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
- 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
- 2022In-context Learning and Induction Headscited by 964
- 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
- 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
16 more
- 2016NewsQA: A Machine Comprehension Datasetcited by 331
- 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
- 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attentioncited by 74
- 2026DeepSeek-OCR 2: Visual Causal Flowcited by 74
- 2025Glyph: Scaling Context Windows via Visual-Text Compressioncited by 49
- 2024Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectlycited by 30
- 2026AgentOCR: Reimagining Agent History via Optical Self-Compressioncited by 22
- 2024Towards Knowledge Checking in Retrieval-augmented Generation: A Representation Perspectivecited by 20
- 2025SelfElicit: Your Language Model Secretly Knows Where is the Relevant Evidencecited by 18
- 2026MemOCR: Layout-Aware Visual Memory for Efficient Long-Horizon Reasoningcited by 15
- 2025VTCBench: Can Vision-Language Models Understand Long Context with Vision-Text Compression?cited by 11
- 2026VTC-R1: Vision-Text Compression for Efficient Long-Context Reasoningcited by 8
- 2025Towards Context-Robust LLMs: A Gated Representation Fine-tuning Approachcited by 6
- 2026Can Vision-Language Models Handle Long-Context Code? An Empirical Study on Visual Compressioncited by 5
- 2026When Text-as-Vision Meets Semantic IDs in Generative Recommendation: An Empirical Studycited by 3
- 2026Attn-GS: Attention-Guided Context Compression for Efficient Personalized LLMscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025ReadBench: Measuring the Dense Text Visual Reading Ability of Vision-Language Models
- 2026Attention-Guided Saliency Maps for Interpreting Visualization Literacy in VLMs
- 2026SimpleOCR: Rendering Visualized Questions to Teach MLLMs to Read
- 2026Layers, Sinks, and Scaling: Adaptive Evidence Selection for Multimodal Large Language Models
- 2026When Vision Becomes Text: Visual Token Pruning via Cross-Modal Residual Guidance in VLMs
- 2026Responses Fall Short of Understanding: Revealing the Gap between Internal Representations and Responses in Visual Document Understanding
- 2026LensVLM: Selective Context Expansion for Compressed Visual Representation of Text
- 2024Cross-Lingual Text-Rich Visual Comprehension: An Information Theory Perspective