# More Accurate, Less Human: Gestalt Grouping in Vision Models

arXiv: https://arxiv.org/abs/2608.10195 · HTML: https://mlsanity.com/paper/2608.10195/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 10 Aug 2026.

## Builds on (most cited first, 22 of 22)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI](https://mlsanity.com/paper/2311.16502/) (2023)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Using cognitive psychology to understand GPT-3](https://mlsanity.com/paper/2206.14576/) (2022)
- [SmolVLM: Redefining small and efficient multimodal models](https://mlsanity.com/paper/2504.05299/) (2025)
- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [Getting aligned on representational alignment](https://mlsanity.com/paper/2310.13018/) (2023)
- [Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency](https://mlsanity.com/paper/2006.16736/) (2020)
- [Human alignment of neural network representations](https://mlsanity.com/paper/2211.01201/) (2022)
- [Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?](https://mlsanity.com/paper/2311.00047/) (2023)
- [Benchmarking Visual Language Models on Standardized Visualization Literacy Tests](https://mlsanity.com/paper/2503.16632/) (2025)
- [From Local Cues to Global Percepts: Emergent Gestalt Organization in Self-Supervised Vision Models](https://mlsanity.com/paper/2506.00718/) (2025)
- [CHART-6: Human-Centered Evaluation of Data Visualization Understanding in Vision-Language Models](https://mlsanity.com/paper/2505.17202/) (2025)
- [Towards Understanding Graphical Perception in Large Multimodal Models](https://mlsanity.com/paper/2503.10857/) (2025)
- [Rethinking the Ranks of Visual Channels](https://mlsanity.com/paper/2107.11367/) (2021)
- [Revisiting Categorical Color Perception in Scatterplots: Sequential, Diverging, and Categorical Palettes](https://mlsanity.com/paper/2404.03787/) (2024)

## Abstract (arXiv)

Human vision organizes what it sees into wholes: same-colored points group into series, similar marks cohere into categories, and shapes complete into recognizable objects. These are the Gestalt operations that visualization design builds on. Whether vision models organize visual content this way has not been systematically tested. We introduce a behavioral battery that scores models against human data from prior perception studies on four grouping tasks: mark-color odd-one-out, color-series counting, silhouette recognition, and object odd-one-out. We apply it to 45 models across five training families: supervised, self-supervised, and contrastive vision-language encoders, open-weight VLMs, and closed foundation models. The battery reveals that agreement with human responses captures aspects of perceptual organization that conventional performance metrics fail to distinguish, with several closed models exhibiting substantially lower alignment than their benchmark accuracy would suggest. Scoring against published perception data therefore gives visualization research a reusable yardstick, requiring no new user study, for auditing whether the models now entering visualization pipelines organize what they see the way their human audience does.