Pay More Attention To Text In High-Resolution MLLMs
Not on the map. Why?
Abstract (arXiv)
Failures of high-resolution MLLMs are commonly attributed to a visual problem, motivating zooming, cropping, and related visual interventions to recover fine-grained evidence or suppress interference. Yet recent studies suggest that relevant visual evidence is already encoded in intermediate representations, indicating that visual-side improvements alone insufficient. This raises a natural question: does the remaining bottleneck lie in the text that guides visual search? We identify a previously overlooked linguistic bottleneck: questions formulated for answering do not necessarily specify the visual evidence required for localization. To address this mismatch, we introduce EviSpec, a training-free compiler that derives complementary evidence specifications while preserving the original question for final reasoning. We further validate it through matched-control experiments that isolate the roles of evidence specification and localization. With the search budget fixed, structured evidence specifications yield an 8.6% relative gain over generic requests. With evidence geometry matched, the evidence localized by EviSpec yields a 14.8% relative gain over random evidence. Together, these controls isolate the benefit of specifying what evidence to seek rather than merely expanding visual access. Across all five MLLMs, EviSpec consistently improves upon the corresponding baseline on each of the three benchmarks, yielding average relative gains of 10.4%, 8.8%, and 12.4% on Vtextsuperscript*Bench, HR-Bench-4K, and HR-Bench-8K, respectively. Beyond high-resolution reasoning, EviSpec also achieves state-of-the-art performance on VQA and hallucination-focused benchmarks.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 56 of its 67 references that are in the map. First posted on arXiv 20 Sep 2026.
Builds on
56 of 67 references.
References · 56
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2024GPT-4o System Cardcited by 4,588
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
38 more
- 2021Grounded Language-Image Pre-trainingcited by 1,156
- 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
- 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
- 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
- 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
- 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 431
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 410
- 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 357
- 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
- 2023Query Rewriting for Retrieval-Augmented Large Language Modelscited by 213
- 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMscited by 176
- 2024Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Modelscited by 161
- 2024InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HDcited by 159
- 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 88
- 2026Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillationcited by 77
- 2024MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuningcited by 76
- 2025VGR: Visual Grounded Reasoningcited by 73
- 2025Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodologycited by 69
- 2026Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perceptioncited by 50
- 2024Towards Semantic Equivalence of Tokenization in Multimodal LLMcited by 45
- 2023Exploring Vision-Language Models for Imbalanced Learningcited by 35
- 2025MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMscited by 33
- 2024Effectiveness Assessment of Recent Large Vision-Language Modelscited by 29
- 2024Mini-Monkey: Alleviating the Semantic Sawtooth Effect for Lightweight MLLMs via Complementary Image Pyramidcited by 26
- 2025AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisitioncited by 21
- 2025FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answeringcited by 20
- 2025HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decouplingcited by 17
- 2026N"uwa: Mending the Spatial Integrity Torn by VLM Token Pruningcited by 15
- 2025Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Visioncited by 12
- 2026Ref-Adv: Exploring MLLM Visual Reasoning in Referring Expression Taskscited by 9
- 2018Question Type Guided Attention in Visual Question Answeringcited by 8
- 2024Visual Question Decomposition on Multimodal Large Language Modelscited by 7
- 2025InfMasking: Unleashing Synergistic Information by Contrastive Multimodal Interactionscited by 7
- 2024ClawMachine: Learning to Fetch Visual Tokens for Referential Comprehensioncited by 6
- 2026DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Modelscited by 6
- 2025PostAlign: Multimodal Grounding as a Corrective Lens for MLLMscited by 5
50 most cited of 56 All 56 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Layers, Sinks, and Scaling: Adaptive Evidence Selection for Multimodal Large Language Models
- 2025Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs
- 2026VisualNeedle: Benchmarking Active Visual Search in Information-Dense Scenes
- 2026VELD-QA: A Large-Scale Dataset for Visual Evidence Grounding and Question Answering in Long Documents
- 2026ProtoLIP: From Sentence-Level to Object-Level Evidence Disentanglement
- 2026Seeing is not Enough: Vision-Language Models Perceive Evidence but Fail to Act
- 2026VERA: Identifying and Leveraging Visual Evidence Retrieval Heads in Long-Context Understanding
- 2026Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning