# Information-Theoretic Segmentation by Inpainting Error Maximization

arXiv: https://arxiv.org/abs/2012.07287 · HTML: https://mlsanity.com/paper/2012.07287/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2020.

## Built on by (most cited first, 22 of 22)

- [Illiterate DALL-E Learns to Compose](https://mlsanity.com/paper/2110.11405/) (2021)
- [Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videos](https://mlsanity.com/paper/2205.14065/) (2022)
- [NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenes](https://mlsanity.com/paper/2209.08776/) (2022)
- [Unsupervised Foreground Extraction via Deep Region Competition](https://mlsanity.com/paper/2110.15497/) (2021)
- [Unsupervised Multi-View Object Segmentation Using Radiance Field Propagation](https://mlsanity.com/paper/2210.00489/) (2022)
- [MOVE: Unsupervised Movable Object Segmentation and Detection](https://mlsanity.com/paper/2210.07920/) (2022)
- [GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generation](https://mlsanity.com/paper/2112.01036/) (2021)
- [AutoLink: Self-supervised Learning of Human Skeletons and Object Outlines by Linking Keypoints](https://mlsanity.com/paper/2205.10636/) (2022)
- [Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Survey](https://mlsanity.com/paper/2310.12904/) (2023)
- [Contrastive Fine-grained Class Clustering via Generative Adversarial Networks](https://mlsanity.com/paper/2112.14971/) (2021)
- [Learning Foreground-Background Segmentation from Improved Layered GANs](https://mlsanity.com/paper/2104.00483/) (2021)
- [Foreground-Background Separation through Concept Distillation from Generative Image Foundation Models](https://mlsanity.com/paper/2212.14306/) (2022)
- [SC-VAE: Sparse Coding-based Variational Autoencoder with Learned ISTA](https://mlsanity.com/paper/2303.16666/) (2023)
- [SEMPART: Self-supervised Multi-resolution Partitioning of Image Semantics](https://mlsanity.com/paper/2309.10972/) (2023)
- [Surface-SOS: Self-Supervised Object Segmentation via Neural Surface Representation](https://mlsanity.com/paper/2501.09947/) (2025)
- [ILSGAN: Independent Layer Synthesis for Unsupervised Foreground-Background Segmentation](https://mlsanity.com/paper/2211.13974/) (2022)
- [Co-Learning Semantic-aware Unsupervised Segmentation for Pathological Image Registration](https://mlsanity.com/paper/2310.11040/) (2023)
- [Unsupervised Part Discovery via Dual Representation Alignment](https://mlsanity.com/paper/2408.08108/) (2024)
- [Factorized Diffusion Architectures for Unsupervised Image Generation and Segmentation](https://mlsanity.com/paper/2309.15726/) (2023)
- [Spectral-Stimulus Information for Self-Supervised Stimulus Encoding](https://mlsanity.com/paper/2407.06195/) (2024)
- [Weakly Supervised Object Segmentation by Background Conditional Divergence](https://mlsanity.com/paper/2506.22505/) (2025)
- [Dialectics for Artificial Intelligence](https://mlsanity.com/paper/2512.17373/) (2025)

## Abstract (arXiv)

We study image segmentation from an information-theoretic perspective, proposing a novel adversarial method that performs unsupervised segmentation by partitioning images into maximally independent sets. More specifically, we group image pixels into foreground and background, with the goal of minimizing predictability of one set from the other. An easily computed loss drives a greedy search process to maximize inpainting error over these partitions. Our method does not involve training deep networks, is computationally cheap, class-agnostic, and even applicable in isolation to a single unlabeled image. Experiments demonstrate that it achieves a new state-of-the-art in unsupervised segmentation quality, while being substantially faster and more general than competing approaches.