mlsanityData to 7 Oct 2026

Labels4Free: Unsupervised Segmentation using StyleGAN

arXiv 2103.14968 ↗First posted 27 Mar 2021cs.CVcited by 53

Not on the map. Why?

Abstract (arXiv)

We propose an unsupervised segmentation framework for StyleGAN generated objects. We build on two main observations. First, the features generated by StyleGAN hold valuable information that can be utilized towards training segmentation networks. Second, the foreground and background can often be treated to be largely independent and be composited in different ways. For our solution, we propose to augment the StyleGAN2 generator architecture with a segmentation branch and to split the generator into a foreground and background network. This enables us to generate soft segmentation masks for the foreground object in an unsupervised fashion. On multiple object classes, we report comparable results against state-of-the-art supervised segmentation networks, while against the best unsupervised segmentation approach we demonstrate a clear improvement, both in qualitative and quantitative metrics.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 40 of its 49 references that are in the map. First posted on arXiv 27 Mar 2021.

Builds on

40 of 49 references.

References · 40

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  4. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  5. 2016Improved Techniques for Training GANscited by 2,938
  6. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  7. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  8. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  9. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  10. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  11. 2019Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?cited by 675
  12. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
28 more
  1. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 624
  2. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  3. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  4. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  5. 2018Invariant Information Clustering for Unsupervised Image Classification and Segmentationcited by 454
  6. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
  7. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  8. 2020InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANscited by 393
  9. 2020PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Modelscited by 358
  10. 2019Image2StyleGAN++: How to Edit the Embedded Images?cited by 356
  11. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 348
  12. 2020StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generationcited by 331
  13. 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 267
  14. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  15. 2020Editing in Style: Uncovering the Local Semantics of GANscited by 183
  16. 2019Seeing What a GAN Cannot Generatecited by 148
  17. 2020Improved StyleGAN Embedding: Where are the Good Latents?cited by 88
  18. 2020Generative Hierarchical Features from Synthesizing Imagescited by 83
  19. 2019Unsupervised Object Segmentation by Redrawingcited by 74
  20. 2018Patch-Based Image Inpainting with Generative Adversarial Networkscited by 70
  21. 2021Intermediate Layer Optimization for Inverse Problems using Deep Generative Modelscited by 70
  22. 2020HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histogramscited by 64
  23. 2020Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANscited by 63
  24. 2020Autoregressive Unsupervised Image Segmentationcited by 60
  25. 2019Emergence of Object Segmentation in Perturbed Generative Modelscited by 57
  26. 2019Detecting Overfitting of Deep Generative Networks via Latent Recoverycited by 44
  27. 2019OneGAN: Simultaneous Unsupervised Learning of Conditional Image Generation, Foreground Segmentation, and Fine-Grained Clusteringcited by 28
  28. 2020Time-Travel Rephotographycited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 16; 2023: 17; 2024: 9; 2025: 4; 2026: 1.

53 papers.

Sort:

By area:

  1. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  2. 2021CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directionscited by 78
  3. 2021Mind the Gap: Domain Gap Control for Single Shot Domain Adaptation for Generative Adversarial Networkscited by 68
  4. 2023Generative Models: What Do They Know? Do They Know Things? Let's Find Out!cited by 50
  5. 2022Region-Aware Face Swappingcited by 45
  6. 2025From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuitcited by 45
  7. 2023StyleGAN knows Normal, Depth, Albedo, and Morecited by 44
  8. 2024EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Modelscited by 42
  9. 2021TransFGU: A Top-down Approach to Fine-Grained Unsupervised Semantic Segmentationcited by 38
  10. 2022State-of-the-Art in the Architecture, Methods and Applications of StyleGANcited by 34
  11. 2022One-Shot Adaptation of GAN in Just One CLIPcited by 31
  12. 2023SmooSeg: Smoothness Prior for Unsupervised Semantic Segmentationcited by 18
38 more
  1. 2022Promising or Elusive? Unsupervised Object Segmentation from Real-world Single Imagescited by 16
  2. 2023U3DS3: Unsupervised 3D Semantic Scene Segmentationcited by 16
  3. 2022Synthetic Data Supervised Salient Object Detectioncited by 15
  4. 2024DeNVeR: Deformable Neural Vessel Representations for Unsupervised Video Vessel Segmentationcited by 13
  5. 2022Bridging CLIP and StyleGAN through Latent Alignment for Image Editingcited by 12
  6. 2023Weakly-supervised Contrastive Learning for Unsupervised Object Discoverycited by 12
  7. 2023On-Device Unsupervised Image Segmentationcited by 10
  8. 2023Progressive Learning of 3D Reconstruction Network from 2D GAN Datacited by 10
  9. 2024How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?cited by 10
  10. 2021ISF-GAN: An Implicit Style Function for High-Resolution Image-to-Image Translationcited by 9
  11. 2021Contrastive Fine-grained Class Clustering via Generative Adversarial Networkscited by 9
  12. 2022HandsOff: Labeled Dataset Generation With No Additional Human Annotationscited by 9
  13. 2021Learning Foreground-Background Segmentation from Improved Layered GANscited by 7
  14. 2023Understanding Self-Supervised Features for Learning Unsupervised Instance Segmentationcited by 7
  15. 2022FurryGAN: High Quality Foreground-aware Image Synthesiscited by 6
  16. 2022Extracting Semantic Knowledge from GANs with Unsupervised Learningcited by 5
  17. 2022Foreground-Background Separation through Concept Distillation from Generative Image Foundation Modelscited by 5
  18. 2023Learning Neural Eigenfunctions for Unsupervised Semantic Segmentationcited by 4
  19. 2023Adversarially-Guided Portrait Mattingcited by 4
  20. 2023Unsupervised Camouflaged Object Segmentation as Domain Adaptationcited by 4
  21. 2023Benchmarking and Analysis of Unsupervised Object Segmentation from Real-world Single Imagescited by 4
  22. 2024Stylized Face Sketch Extraction via Generative Prior with Limited Datacited by 4
  23. 2024DynaSeg: A Deep Dynamic Fusion Method for Unsupervised Image Segmentation Incorporating Feature Similarity and Spatial Continuitycited by 4
  24. 2022Unveiling the Latent Space Geometry of Push-Forward Generative Modelscited by 3
  25. 2023Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentationcited by 3
  26. 2024Revisiting Surgical Instrument Segmentation Without Human Intervention: A Graph Partitioning Viewcited by 3
  27. 2022ILSGAN: Independent Layer Synthesis for Unsupervised Foreground-Background Segmentationcited by 2
  28. 2023Self-Supervised One-Shot Learning for Automatic Segmentation of StyleGAN Imagescited by 2
  29. 2023FreeCOS: Self-Supervised Learning from Fractals and Unlabeled Images for Curvilinear Object Segmentationcited by 2
  30. 2024Composing Parts for Expressive Object Generationcited by 2
  31. 2022Interpreting Class Conditional GANs with Channel Awarenesscited by 1
  32. 2022S2-Flow: Joint Semantic and Style Editing of Facial Imagescited by 1
  33. 2023SS-CPGAN: Self-Supervised Cut-and-Pasting Generative Adversarial Network for Object Segmentationcited by 1
  34. 2025SketchYourSeg: Mask-Free Subjective Image Segmentation via Freehand Sketchescited by 1
  35. 2025FFaceNeRF: Few-shot Face Editing in Neural Radiance Fieldscited by 1
  36. 2022Learning Representations for Masked Facial Recoverycited by 0
  37. 2023Explainable Semantic Medical Image Segmentation with Stylecited by 0
  38. 2024Leveraging Activations for Superpixel Explanationscited by 0

50 most cited of 53 All 53 (JSON)