# Disentangling neural mechanisms for perceptual grouping

arXiv: https://arxiv.org/abs/1906.01558 · HTML: https://mlsanity.com/paper/1906.01558/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2019.

## Built on by (most cited first, 28 of 28)

- [What Makes Convolutional Models Great on Long Sequence Modeling?](https://mlsanity.com/paper/2210.09298/) (2022)
- [Harmonizing the object recognition strategies of deep neural networks with humans](https://mlsanity.com/paper/2211.04533/) (2022)
- [A Benchmark for Compositional Visual Reasoning](https://mlsanity.com/paper/2206.05379/) (2022)
- [Neither hype nor gloom do DNNs justice](https://mlsanity.com/paper/2312.05355/) (2023)
- [Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors](https://mlsanity.com/paper/2310.02980/) (2023)
- [Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks](https://mlsanity.com/paper/2201.11729/) (2022)
- [Simplifying and Understanding State Space Models with Diagonal Linear RNNs](https://mlsanity.com/paper/2212.00768/) (2022)
- [BayesFormer: Transformer with Uncertainty Estimation](https://mlsanity.com/paper/2206.00826/) (2022)
- [The 3D-PC: a benchmark for visual perspective taking in humans and machines](https://mlsanity.com/paper/2406.04138/) (2024)
- [Tracking Without Re-recognition in Humans and Machines](https://mlsanity.com/paper/2105.13351/) (2021)
- [Better artificial intelligence does not mean better models of biology](https://mlsanity.com/paper/2504.16940/) (2025)
- [PQA: Perceptual Question Answering](https://mlsanity.com/paper/2104.03589/) (2021)
- [Affinity-based Attention in Self-supervised Transformers Predicts Dynamics of Object Grouping in Humans](https://mlsanity.com/paper/2306.00294/) (2023)
- [Spectraformer: A Unified Random Feature Framework for Transformer](https://mlsanity.com/paper/2405.15310/) (2024)
- [Perceptual Group Tokenizer: Building Perception with Iterative Grouping](https://mlsanity.com/paper/2311.18296/) (2023)
- [Visual Attention Emerges from Recurrent Sparse Reconstruction](https://mlsanity.com/paper/2204.10962/) (2022)
- [Measuring uncertainty in human visual segmentation](https://mlsanity.com/paper/2301.07807/) (2023)
- [Tracking objects that change in appearance with phase synchrony](https://mlsanity.com/paper/2410.02094/) (2024)
- [Recurrent Attention Models with Object-centric Capsule Representation for Multi-object Recognition](https://mlsanity.com/paper/2110.04954/) (2021)
- [FISBe: A real-world benchmark dataset for instance segmentation of long-range thin filamentous structures](https://mlsanity.com/paper/2404.00130/) (2024)
- [Pretrained Hybrids with MAD Skills](https://mlsanity.com/paper/2406.00894/) (2024)
- [Beyond the limitations of any imaginable mechanism: large language models and psycholinguistics](https://mlsanity.com/paper/2303.00077/) (2023)
- [Human-like Object Grouping in Self-supervised Vision Transformers](https://mlsanity.com/paper/2603.13994/) (2026)
- [I Walk the Line: Examining the Role of Gestalt Continuity in Object Binding for Vision Transformers](https://mlsanity.com/paper/2604.09942/) (2026)
- [Do Neural Networks for Segmentation Understand Insideness?](https://mlsanity.com/paper/2201.10664/) (2022)
- [Diagnosing and exploiting the computational demands of videos games for deep reinforcement learning](https://mlsanity.com/paper/2309.13181/) (2023)
- [Fixing the problems of deep neural networks will require better training data and learning algorithms](https://mlsanity.com/paper/2311.12819/) (2023)
- [Converting Transformers into DGNNs Form](https://mlsanity.com/paper/2502.00585/) (2025)

## Abstract (arXiv)

Forming perceptual groups and individuating objects in visual scenes is an essential step towards visual intelligence. This ability is thought to arise in the brain from computations implemented by bottom-up, horizontal, and top-down connections between neurons. However, the relative contributions of these connections to perceptual grouping are poorly understood. We address this question by systematically evaluating neural network architectures featuring combinations bottom-up, horizontal, and top-down connections on two synthetic visual tasks, which stress low-level "Gestalt" vs. high-level object cues for perceptual grouping. We show that increasing the difficulty of either task strains learning for networks that rely solely on bottom-up connections. Horizontal connections resolve straining on tasks with Gestalt cues by supporting incremental grouping, whereas top-down connections rescue learning on tasks with high-level object cues by modifying coarse predictions about the position of the target object. Our findings dissociate the computational roles of bottom-up, horizontal and top-down connectivity, and demonstrate how a model featuring all of these interactions can more flexibly learn to form perceptual groups.