# Latent Noise Segmentation: How Neural Noise Leads to the Emergence of Segmentation and Grouping

arXiv: https://arxiv.org/abs/2309.16515 · HTML: https://mlsanity.com/paper/2309.16515/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 93 references that are in the map.
- First posted on arXiv 28 Sep 2023.

## Builds on (most cited first, 30 of 34)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [Unsupervised Semantic Segmentation by Distilling Feature Correspondences](https://mlsanity.com/paper/2203.08414/) (2022)
- [Generalisation in humans and deep neural networks](https://mlsanity.com/paper/1808.08750/) (2018)
- [On the Binding Problem in Artificial Neural Networks](https://mlsanity.com/paper/2012.05208/) (2020)
- [GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representations](https://mlsanity.com/paper/1907.13052/) (2019)
- [How to represent part-whole hierarchies in a neural network](https://mlsanity.com/paper/2102.12627/) (2021)
- [GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinement](https://mlsanity.com/paper/2104.09958/) (2021)
- [Learning long-range spatial dependencies with horizontal gated-recurrent units](https://mlsanity.com/paper/1805.08315/) (2018)
- [Taming VAEs](https://mlsanity.com/paper/1810.00597/) (2018)
- [Variational Autoencoders Pursue PCA Directions (by Accident)](https://mlsanity.com/paper/1812.06775/) (2018)
- [Complex-Valued Autoencoders for Object Discovery](https://mlsanity.com/paper/2204.02075/) (2022)
- [From Principal Subspaces to Principal Components with Linear Autoencoders](https://mlsanity.com/paper/1804.10253/) (2018)
- [Unsupervised deep learning identifies semantic disentanglement in single inferotemporal neurons](https://mlsanity.com/paper/2006.14304/) (2020)
- [The Free Energy Principle for Perception and Action: A Deep Learning Perspective](https://mlsanity.com/paper/2207.06415/) (2022)
- [Predictive Coding, Variational Autoencoders, and Biological Connections](https://mlsanity.com/paper/2011.07464/) (2020)
- [Unifying (Machine) Vision via Counterfactual World Modeling](https://mlsanity.com/paper/2306.01828/) (2023)
- [Reconstruction Bottlenecks in Object-Centric Generative Models](https://mlsanity.com/paper/2007.06245/) (2020)
- [Contrastive Training of Complex-Valued Autoencoders for Object Discovery](https://mlsanity.com/paper/2305.15001/) (2023)
- [Mixed Evidence for Gestalt Grouping in Deep Neural Networks](https://mlsanity.com/paper/2203.07302/) (2022)

## Built on by (most cited first, 4 of 4)

- [MindSet: Vision. A toolbox for testing DNNs on key psychological experiments](https://mlsanity.com/paper/2404.05290/) (2024)
- [Contour Integration Underlies Human-Like Vision](https://mlsanity.com/paper/2504.05253/) (2025)
- [Finding Closure: A Closer Look at the Gestalt Law of Closure in Convolutional Neural Networks](https://mlsanity.com/paper/2408.12460/) (2024)
- [Demystifying Variance in Circuit Discovery of LLMs](https://mlsanity.com/paper/2606.16920/) (2026)

## Abstract (arXiv)

Humans are able to segment images effortlessly without supervision using perceptual grouping. Here, we propose a counter-intuitive computational approach to solving unsupervised perceptual grouping and segmentation: that they arise because of neural noise, rather than in spite of it. We (1) mathematically demonstrate that under realistic assumptions, neural noise can be used to separate objects from each other; (2) that adding noise in a DNN enables the network to segment images even though it was never trained on any segmentation labels; and (3) that segmenting objects using noise results in segmentation performance that aligns with the perceptual grouping phenomena observed in humans, and is sample-efficient. We introduce the Good Gestalt (GG) datasets -- six datasets designed to specifically test perceptual grouping, and show that our DNN models reproduce many important phenomena in human perception, such as illusory contours, closure, continuity, proximity, and occlusion. Finally, we (4) show that our model improves performance on our GG datasets compared to other tested unsupervised models by 24.9%. Together, our results suggest a novel unsupervised segmentation method requiring few assumptions, a new explanation for the formation of perceptual grouping, and a novel potential benefit of neural noise.