Matched Outcomes, Divergent Gaze: How Foveated MLLMs Search Compared to Humans
Not on the map. Why?
Abstract (arXiv)
Human visual search is serial: the fovea must land on a candidate to confirm it, and those landings form a scanpath. Whether multimodal large language models (MLLMs), given the same foveated input, search as humans do bears on their use as models of human vision and on attention-alignment scores. We compare three general-purpose MLLMs with human eye-movement scanpaths on goal-directed search (COCO-Search18), driving each model fixation by fixation through an identical, human-matched foveated view and assessing it along three axes: the decision of target presence, the efficiency of reaching the target, and the gaze process itself. The axes dissociate. On the decision and on target acquisition the models match or exceed humans, detecting present targets near ceiling and reaching them on the first saccade more often than people do. The gaze process is not human. Under the human-matched condition, all three share one signature: low-entropy, large-amplitude, self-consistent scanpaths that agree with themselves far more closely than two humans agree with each other. That is consistent with a single-pass, non-serial architecture rather than a limit of acuity. Matched retinal input reproduces where humans look but not how the looking unfolds in time, and no degradation regime recovers human-like search at human-like success. The gap sits on a process axis that answer-alignment and saliency metrics do not measure. Because they miss it, such metrics cannot certify human-like vision, and zero-shot models suit outcome and spatial questions but not temporal, process-level ones.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 39 of its 59 references that are in the map. First posted on arXiv 17 Aug 2026.
Builds on
39 of 59 references.
References · 39
- 2022Token Merging: Your ViT But Fastercited by 1,021
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
- 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 466
- 2025MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMscited by 176
- 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 120
- 2024ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Explorationcited by 103
- 2020Predicting Goal-directed Human Attention Using Inverse Reinforcement Learningcited by 53
- 2023Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attentioncited by 38
- 2020Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehensioncited by 29
- 2021FoveaTer: Foveated Transformer for Image Classificationcited by 28
- 2020Emergent Properties of Foveated Perceptual Systemscited by 24
27 more
- 2023Unifying Top-down and Bottom-up Scanpath Prediction Using Transformerscited by 22
- 2021VQA-MHUG: A Gaze Dataset to Study Multimodal Neural Attention in Visual Question Answeringcited by 16
- 2025Thinking in 360deg: Humanoid Visual Search in the Wildcited by 15
- 2021Visual Search Asymmetry: Deep Nets and Humans Share Similar Inherent Biasescited by 12
- 2025From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMscited by 12
- 2025Do You See Me : A Multidimensional Benchmark for Evaluating Visual Perception in Multimodal LLMscited by 11
- 2025InSight-o3: Empowering Multimodal Foundation Models with Generalized Visual Searchcited by 9
- 2017Central and peripheral vision for scene recognition: A neurocomputational modeling explorationcited by 8
- 2022Behind the Machine's Gaze: Neural Networks with Biologically-inspired Constraints Exhibit Human-like Visual Attentioncited by 7
- 2025VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMscited by 7
- 2022A domain adaptive deep learning solution for scanpath prediction of paintingscited by 6
- 2025VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation agentscited by 5
- 2020Foveated Model Observers for Visual Search in 3D Medical Imagescited by 4
- 2022SalyPath360: Saliency and Scanpath Prediction Framework for Omnidirectional Imagescited by 4
- 2023Contrastive Language-Image Pretrained Models are Zero-Shot Human Scanpath Predictorscited by 4
- 2024MLLM-Search: A Zero-Shot Approach to Finding People using Multimodal Large Language Modelscited by 4
- 2025Visual serial processing deficits explain divergences in human and VLM reasoningcited by 4
- 2026Glance-or-Gaze: Incentivizing LMMs to Adaptively Focus Search via Reinforcement Learningcited by 4
- 2022An Inter-observer consistent deep adversarial training for visual scanpath predictioncited by 2
- 2023Learning to search for and detect objects in foveal images using deep learningcited by 2
- 2024Do MLLMs Exhibit Human-like Perceptual Behaviors? HVSBench: A Benchmark for MLLM Alignment with Human Perceptual Behaviorcited by 2
- 2025Human Scanpath Prediction in Target-Present Visual Search with Semantic-Foveal Bayesian Attentioncited by 2
- 2025I Spy With My Model's Eye: Visual Search as a Behavioural Test for MLLMscited by 2
- 2026What They Saw, Not Just Where They Looked: Semantic Scanpath Similarity via VLMs and NLP metriccited by 2
- 2024Semantic-Based Active Perception for Humanoid Visual Tasks with Foveal Sensorscited by 1
- 2025Predicting Reaction Time to Comprehend Scenes with Foveated Scene Understanding Mapscited by 1
- 2026SPGen: Stochastic scanpath generation for paintings using unsupervised domain adaptationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Human-Like Attention? A Psychophysical Comparison of Visual Search in Humans and MLLMs
- 2026Emergent Goal-Directed Attention in Large Vision-Language Models
- 2026An Exam for Active Observers
- 2026Do vision-language models search like humans? Reasoning tokens as a reaction-time analog in classic visual-search paradigms
- 2026Similar Choices, Different Attention: Cross-Modal Associations in Humans and Vision-Language Models
- 2022Target-absent Human Attention
- 2026Why We Look Where We Look: Emergent Human-like Fixations of a Foveated Visual Language Model Maximizing Scene Understanding
- 2026Eyes on VLM: Benchmarking Gaze Following and Social Gaze Prediction in Vision Language Models