# Matched Outcomes, Divergent Gaze: How Foveated MLLMs Search Compared to Humans

arXiv: https://arxiv.org/abs/2608.16514 · HTML: https://mlsanity.com/paper/2608.16514/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 59 references that are in the map.
- First posted on arXiv 17 Aug 2026.

## Builds on (most cited first, 30 of 39)

- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [BLINK: Multimodal Large Language Models Can See but Not Perceive](https://mlsanity.com/paper/2404.12390/) (2024)
- [V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs](https://mlsanity.com/paper/2312.14135/) (2023)
- [GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning](https://mlsanity.com/paper/2507.01006/) (2025)
- [MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs](https://mlsanity.com/paper/2502.17422/) (2025)
- [Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search](https://mlsanity.com/paper/2509.07969/) (2025)
- [ZoomEye: Enhancing Multimodal LLMs with Human-Like Zooming Capabilities through Tree-Based Image Exploration](https://mlsanity.com/paper/2411.16044/) (2024)
- [Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning](https://mlsanity.com/paper/2005.14310/) (2020)
- [Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention](https://mlsanity.com/paper/2303.15274/) (2023)
- [Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension](https://mlsanity.com/paper/2010.06396/) (2020)
- [FoveaTer: Foveated Transformer for Image Classification](https://mlsanity.com/paper/2105.14173/) (2021)
- [Emergent Properties of Foveated Perceptual Systems](https://mlsanity.com/paper/2006.07991/) (2020)
- [Unifying Top-down and Bottom-up Scanpath Prediction Using Transformers](https://mlsanity.com/paper/2303.09383/) (2023)
- [VQA-MHUG: A Gaze Dataset to Study Multimodal Neural Attention in Visual Question Answering](https://mlsanity.com/paper/2109.13116/) (2021)
- [Thinking in 360deg: Humanoid Visual Search in the Wild](https://mlsanity.com/paper/2511.20351/) (2025)
- [Visual Search Asymmetry: Deep Nets and Humans Share Similar Inherent Biases](https://mlsanity.com/paper/2106.02953/) (2021)
- [From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs](https://mlsanity.com/paper/2512.19683/) (2025)
- [Do You See Me : A Multidimensional Benchmark for Evaluating Visual Perception in Multimodal LLMs](https://mlsanity.com/paper/2506.02022/) (2025)
- [InSight-o3: Empowering Multimodal Foundation Models with Generalized Visual Search](https://mlsanity.com/paper/2512.18745/) (2025)
- [Central and peripheral vision for scene recognition: A neurocomputational modeling exploration](https://mlsanity.com/paper/1705.00816/) (2017)
- [Behind the Machine's Gaze: Neural Networks with Biologically-inspired Constraints Exhibit Human-like Visual Attention](https://mlsanity.com/paper/2204.09093/) (2022)
- [VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMs](https://mlsanity.com/paper/2507.13361/) (2025)
- [A domain adaptive deep learning solution for scanpath prediction of paintings](https://mlsanity.com/paper/2209.11338/) (2022)
- [VLN-MME: Diagnosing MLLMs as Language-guided Visual Navigation agents](https://mlsanity.com/paper/2512.24851/) (2025)
- [Foveated Model Observers for Visual Search in 3D Medical Images](https://mlsanity.com/paper/2011.01405/) (2020)
- [SalyPath360: Saliency and Scanpath Prediction Framework for Omnidirectional Images](https://mlsanity.com/paper/2201.00096/) (2022)
- [Contrastive Language-Image Pretrained Models are Zero-Shot Human Scanpath Predictors](https://mlsanity.com/paper/2305.12380/) (2023)
- [MLLM-Search: A Zero-Shot Approach to Finding People using Multimodal Large Language Models](https://mlsanity.com/paper/2412.00103/) (2024)
- [Visual serial processing deficits explain divergences in human and VLM reasoning](https://mlsanity.com/paper/2509.25142/) (2025)
- [Glance-or-Gaze: Incentivizing LMMs to Adaptively Focus Search via Reinforcement Learning](https://mlsanity.com/paper/2601.13942/) (2026)

## Abstract (arXiv)

Human visual search is serial: the fovea must land on a candidate to confirm it, and those landings form a scanpath. Whether multimodal large language models (MLLMs), given the same foveated input, search as humans do bears on their use as models of human vision and on attention-alignment scores. We compare three general-purpose MLLMs with human eye-movement scanpaths on goal-directed search (COCO-Search18), driving each model fixation by fixation through an identical, human-matched foveated view and assessing it along three axes: the decision of target presence, the efficiency of reaching the target, and the gaze process itself. The axes dissociate. On the decision and on target acquisition the models match or exceed humans, detecting present targets near ceiling and reaching them on the first saccade more often than people do. The gaze process is not human. Under the human-matched condition, all three share one signature: low-entropy, large-amplitude, self-consistent scanpaths that agree with themselves far more closely than two humans agree with each other. That is consistent with a single-pass, non-serial architecture rather than a limit of acuity. Matched retinal input reproduces where humans look but not how the looking unfolds in time, and no degradation regime recovers human-like search at human-like success. The gap sits on a process axis that answer-alignment and saliency metrics do not measure. Because they miss it, such metrics cannot certify human-like vision, and zero-shot models suit outcome and spatial questions but not temporal, process-level ones.