# GazeVLM: Active Vision via Internal Attention Control for Multimodal Reasoning arXiv: https://arxiv.org/abs/2605.07817 ยท HTML: https://mlsanity.com/paper/2605.07817/ ## Key facts - No citing papers in the map yet (data through 14 Oct 2026). - Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. - Builds on 37 of its 44 references that are in the map. - First posted on arXiv 8 May 2026. ## Placed in - LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/) ## Builds on (most cited first, 30 of 37) - [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024) - [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025) - [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023) - [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024) - [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025) - [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023) - [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023) - [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025) - [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023) - [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023) - [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022) - [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017) - [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023) - [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024) - [Are We on the Right Way for Evaluating Large Vision-Language Models?](https://mlsanity.com/paper/2403.20330/) (2024) - [Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs](https://mlsanity.com/paper/2406.16860/) (2024) - [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023) - [Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding](https://mlsanity.com/paper/2311.16922/) (2023) - [Object Hallucination in Image Captioning](https://mlsanity.com/paper/1809.02156/) (2018) - [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025) - [Ferret: Refer and Ground Anything Anywhere at Any Granularity](https://mlsanity.com/paper/2310.07704/) (2023) - [V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs](https://mlsanity.com/paper/2312.14135/) (2023) - [InfographicVQA](https://mlsanity.com/paper/2104.12756/) (2021) - [DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning](https://mlsanity.com/paper/2505.14362/) (2025) - [PlotQA: Reasoning over Scientific Plots](https://mlsanity.com/paper/1909.00997/) (2019) - [Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning](https://mlsanity.com/paper/2505.15966/) (2025) - [LLaVA-OneVision-1.5: Fully Open Framework for Democratized Multimodal Training](https://mlsanity.com/paper/2509.23661/) (2025) - [Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models](https://mlsanity.com/paper/2408.15556/) (2024) - [Online Difficulty Filtering for Reasoning Oriented Reinforcement Learning](https://mlsanity.com/paper/2504.03380/) (2025) - [Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs](https://mlsanity.com/paper/2505.15436/) (2025) ## Abstract (arXiv) Human visual reasoning is governed by active vision, a process where meta-cognitive control drives top-down goal-directed attention, dynamically routing foveal focus toward task-relevant details while maintaining peripheral awareness of the global scene. In contrast, modern Vision-Language Models (VLMs) process visual information passively, relying on the static accumulation of massive token contexts that dilute the visual evidence as the reasoning chain grows. Here we propose GazeVLM, a VLM that learns to exert this oversight over its attention resources by generating gaze actions in its reasoning chain, in the form of tags specifying the coordinates of the region to inspect. During training, each block triggers a continuous suppression bias on the attention logits that dampens features outside the regions gazed at so far, and that lifts at the end of the block, restoring the global view. The model is trained with this bias as a teaching signal, first by Supervised Fine-Tuning on curated gaze-reasoning traces, then through Group Relative Policy Optimization (GRPO) with rewards for correct answers and valid grounding. At deployment, the external suppression is removed and the model, running its unmodified forward pass, steers its own attention to the regions indicated by the blocks, retaining most of the effect of the bias and emulating top-down control of spatial attention. This model thus transitions between global spatial awareness and localized focal reasoning without relying on external agentic tools like cropping, and without adding visual tokens from re-encoded patches. Applied to 4B-parameter backbones, GazeVLM demonstrates strong high-resolution multimodal reasoning on HRBench-4k and HRBench-8k, surpassing its base models by about 4 points, and outperforming agentic multimodal pipelines built around thinking with images by 5 to 12 points.