mlsanityData to 14 Oct 2026

Emergent Goal-Directed Attention in Large Vision-Language Models

arXiv 2609.05517 ↗First posted 31 Aug 2026cs.CV cs.AI cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Human observers prioritize visual information according to task goals. Most computational models of naturalistic viewing are gaze-trained for free viewing, leaving open whether goal-directed attention can emerge in systems without gaze supervision. We tested two off-the-shelf vision-language models (VLMs), Qwen3-VL-32B-Thinking and Gemma-4-26B-A4B-it, on 4,887 naturalistic scenes under visual-search and free-viewing instructions. Model predictions were compared with human fixations on the same images under corresponding tasks. Both models aligned more closely with human fixations under matching goals than under mismatched goals. This crossover persisted in target-absent scenes, where alignment could not be explained by simple visual grounding, and appeared in decoder-layer readouts. Furthermore, model-thinking traces were grounded in target semantics during search and in visual prominence during free viewing. These findings show that general-purpose VLMs can generate human-aligned, goal-directed spatial priorities without gaze-specific training, informing theories of goal-directed attention and offering scalable tools for predicting where people look across tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 9 of its 30 references that are in the map. First posted on arXiv 31 Aug 2026.

Builds on

9 of 30 references.

References · 9

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2025Qwen3-VL Technical Reportcited by 2,793
  3. 2026Gemma 4 Technical Reportcited by 259
  4. 2022Emergent Analogical Reasoning in Large Language Modelscited by 240
  5. 2023Playing repeated games with Large Language Modelscited by 217
  6. 2016What do different evaluation metrics tell us about saliency models?cited by 185
  7. 2016DeepGaze II: Reading fixations from deep features trained on object recognitioncited by 55
  8. 2023Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attentioncited by 38
  9. 2021SalFBNet: Learning Pseudo-Saliency Distribution via Feedback Convolutional Networkscited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Why We Look Where We Look: Emergent Human-like Fixations of a Foveated Visual Language Model Maximizing Scene Understanding
  2. 2026Gaze-to-text Generation: Beyond Categorical Decoding of Human Attention
  3. 2026Gaze Heads: How VLMs Look at What They Describe
  4. 2023Attention, not scale, drives human-AI alignment in multimodal language prediction
  5. 2026Matched Outcomes, Divergent Gaze: How Foveated MLLMs Search Compared to Humans
  6. 2025CoFFT: Chain of Foresight-Focus Thought for Visual Language Models
  7. 2026GazeVLM: Active Vision via Internal Attention Control for Multimodal Reasoning
  8. 2026Visual Attention Faithfulness in Vision-Language Models is Heterogeneous