mlsanityData to 7 Oct 2026

Finding an Unsupervised Image Segmenter in Each of Your Deep Generative Models

arXiv 2105.08127 ↗First posted 17 May 2021cs.CV cs.AIcited by 44

Not on the map. Why?

Abstract (arXiv)

Recent research has shown that numerous human-interpretable directions exist in the latent space of GANs. In this paper, we develop an automatic procedure for finding directions that lead to foreground-background image separation, and we use these directions to train an image segmentation model without human supervision. Our method is generator-agnostic, producing strong segmentation results with a wide range of different GAN architectures. Furthermore, by leveraging GANs pretrained on large datasets such as ImageNet, we are able to segment images from a range of domains without further training or finetuning. Evaluating our method on image segmentation benchmarks, we compare favorably to prior work while using neither human supervision nor access to the training data. Broadly, our results demonstrate that automatically extracting foreground-background structure from pretrained deep generative models can serve as a remarkably effective substitute for human supervision.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 39 of its 60 references that are in the map. First posted on arXiv 17 May 2021.

Builds on

39 of 60 references.

References · 39

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  7. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,079
  8. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  9. 2018Self-Attention Generative Adversarial Networkscited by 809
  10. 2018ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentationcited by 623
  11. 2018Learning to Adapt Structured Output Space for Semantic Segmentationcited by 584
  12. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 562
27 more
  1. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 454
  2. 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
  3. 2020Unsupervised Discovery of Interpretable Directions in the GAN Latent Spacecited by 288
  4. 2016Learning from Simulated and Unsupervised Images through Adversarial Trainingcited by 287
  5. 2019Large Scale Adversarial Representation Learningcited by 238
  6. 2017Geometric GANcited by 232
  7. 2016Deeply supervised salient object detection with short connectionscited by 196
  8. 2019Seeing What a GAN Cannot Generatecited by 148
  9. 2017Amulet: Aggregating Multi-level Convolutional Features for Salient Object Detectioncited by 93
  10. 2020The Hessian Penalty: A Weak Prior for Unsupervised Disentanglementcited by 90
  11. 2020Unsupervised Domain Adaptation in Semantic Segmentation: a Reviewcited by 88
  12. 2016Disentangling factors of variation in deep representations using adversarial trainingcited by 87
  13. 2019Unsupervised Object Segmentation by Redrawingcited by 74
  14. 2020Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANscited by 63
  15. 2020Autoregressive Unsupervised Image Segmentationcited by 60
  16. 2017W-Net: A Deep Model for Fully Unsupervised Image Segmentationcited by 57
  17. 2019Emergence of Object Segmentation in Perturbed Generative Modelscited by 57
  18. 2020Diverse Image Generation via Self-Conditioned GANscited by 54
  19. 2018Deep Unsupervised Saliency Detection: A Multiple Noisy Labeling Perspectivecited by 50
  20. 2017Learning Uncertain Convolutional Features for Accurate Saliency Detectioncited by 44
  21. 2019Multi-source weak supervision for saliency detectioncited by 43
  22. 2019DeepUSPS: Deep Robust Unsupervised Saliency Prediction With Self-Supervisioncited by 38
  23. 2020Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Datacited by 30
  24. 2019Object Discovery with a Copy-Pasting GANcited by 29
  25. 2019OneGAN: Simultaneous Unsupervised Learning of Conditional Image Generation, Foreground Segmentation, and Fine-Grained Clusteringcited by 28
  26. 2015DeepSaliency: Multi-Task Deep Neural Network Model for Salient Object Detectioncited by 24
  27. 2017Disentangling Factors of Variation by Mixing Themcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 15; 2023: 13; 2024: 6; 2025: 3; 2026: 1.

44 papers.

Sort:

By area:

  1. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  2. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  3. 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
  4. 2021ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentationcited by 93
  5. 2022Unsupervised Object Localization: Observing the Background to Discover Objectscited by 66
  6. 2023Diffusion Models for Open-Vocabulary Segmentationcited by 63
  7. 2021GAN-Supervised Dense Visual Alignmentcited by 61
  8. 2022Improving Object-centric Learning with Query Optimizationcited by 60
  9. 2021Unsupervised Part Discovery from Contrastive Reconstructioncited by 57
  10. 2022Unsupervised Salient Object Detection with Spectral Cluster Votingcited by 56
  11. 2023DiffusionSeg: Adapting Diffusion Towards Unsupervised Object Discoverycited by 53
  12. 2022Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motioncited by 42
32 more
  1. 2022ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentationcited by 42
  2. 2024EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Modelscited by 42
  3. 2024Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Modelscited by 32
  4. 2023LD-ZNet: A Latent Diffusion Approach for Text-Based Image Segmentationcited by 29
  5. 2022PandA: Unsupervised Learning of Parts and Appearances in the Feature Maps of GANscited by 20
  6. 2022MOVE: Unsupervised Movable Object Segmentation and Detectioncited by 20
  7. 2023CHORUS: Learning Canonicalized 3D Human-Object Spatial Relations from Unbounded Synthesized Imagescited by 19
  8. 2023Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Surveycited by 16
  9. 2021Data synthesis and adversarial networks: A review and meta-analysis in cancer imagingcited by 15
  10. 2022Synthetic Data Supervised Salient Object Detectioncited by 15
  11. 2023Weakly-supervised Contrastive Learning for Unsupervised Object Discoverycited by 12
  12. 2021Contrastive Fine-grained Class Clustering via Generative Adversarial Networkscited by 9
  13. 2023LOCATE: Self-supervised Object Discovery via Flow-guided Graph-cut and Bootstrapped Self-trainingcited by 8
  14. 2023PaintSeg: Training-free Segmentation via Paintingcited by 7
  15. 2022FurryGAN: High Quality Foreground-aware Image Synthesiscited by 6
  16. 2022Refine and Represent: Region-to-Object Representation Learningcited by 5
  17. 2022Extracting Semantic Knowledge from GANs with Unsupervised Learningcited by 5
  18. 2022Foreground-Background Separation through Concept Distillation from Generative Image Foundation Modelscited by 5
  19. 2022Interpreting Latent Spaces of Generative Models for Medical Images using Unsupervised Methodscited by 4
  20. 2023SEMPART: Self-supervised Multi-resolution Partitioning of Image Semanticscited by 4
  21. 2024DynaSeg: A Deep Dynamic Fusion Method for Unsupervised Image Segmentation Incorporating Feature Similarity and Spatial Continuitycited by 4
  22. 2025DAViD: Modeling Dynamic Affordance of 3D Objects Using Pre-trained Video Diffusion Modelscited by 4
  23. 2022ILSGAN: Independent Layer Synthesis for Unsupervised Foreground-Background Segmentationcited by 2
  24. 2024SimSAM: Simple Siamese Representations Based Semantic Affinity Matrix for Unsupervised Image Segmentationcited by 2
  25. 2023SS-CPGAN: Self-Supervised Cut-and-Pasting Generative Adversarial Network for Object Segmentationcited by 1
  26. 2025SketchYourSeg: Mask-Free Subjective Image Segmentation via Freehand Sketchescited by 1
  27. 2023Unsupervised Cardiac Segmentation Utilizing Synthesized Images from Anatomical Labelscited by 0
  28. 2023A Sparse Graph Formulation for Efficient Spectral Image Segmentationcited by 0
  29. 2024Joint-Embedding Predictive Architecture for Self-Supervised Learning of Mask Classification Architecturecited by 0
  30. 2024PEEKABOO: Hiding parts of an image for unsupervised object localizationcited by 0
  31. 2025A Mutual Information Perspective on Multiple Latent Variable Generative Models for Positive View Generationcited by 0
  32. 2026PANC: Prior-Aware Normalized Cut via Anchor-Augmented Token Graphscited by 0