# Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP

arXiv: https://arxiv.org/abs/2107.12518 · HTML: https://mlsanity.com/paper/2107.12518/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 26 Jul 2021.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs](https://mlsanity.com/paper/2005.09635/) (2020)
- [Dilated Residual Networks](https://mlsanity.com/paper/1705.09914/) (2017)
- [DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort](https://mlsanity.com/paper/2104.06490/) (2021)
- [Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalization](https://mlsanity.com/paper/2104.05833/) (2021)
- [StyleGAN2 Distillation for Feed-forward Image Manipulation](https://mlsanity.com/paper/2003.03581/) (2020)
- [OpenEDS2020: Open Eyes Dataset](https://mlsanity.com/paper/2005.03876/) (2020)

## Built on by (most cited first, 21 of 21)

- [CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation](https://mlsanity.com/paper/2110.02624/) (2021)
- [What does CLIP know about a red circle? Visual prompt engineering for VLMs](https://mlsanity.com/paper/2304.06712/) (2023)
- [ZegCLIP: Towards Adapting CLIP for Zero-shot Semantic Segmentation](https://mlsanity.com/paper/2212.03588/) (2022)
- [LAFITE: Towards Language-Free Training for Text-to-Image Generation](https://mlsanity.com/paper/2111.13792/) (2021)
- [CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP](https://mlsanity.com/paper/2110.11316/) (2021)
- [Open-world Semantic Segmentation via Contrasting and Clustering Vision-Language Embedding](https://mlsanity.com/paper/2207.08455/) (2022)
- [Renderable Neural Radiance Map for Visual Navigation](https://mlsanity.com/paper/2303.00304/) (2023)
- [CrowdCLIP: Unsupervised Crowd Counting via Vision-Language Model](https://mlsanity.com/paper/2304.04231/) (2023)
- [LD-ZNet: A Latent Diffusion Approach for Text-Based Image Segmentation](https://mlsanity.com/paper/2303.12343/) (2023)
- [Spectral Prompt Tuning:Unveiling Unseen Classes for Zero-Shot Semantic Segmentation](https://mlsanity.com/paper/2312.12754/) (2023)
- [CoralStyleCLIP: Co-optimized Region and Layer Selection for Image Editing](https://mlsanity.com/paper/2303.05031/) (2023)
- [Fantastic Style Channels and Where to Find Them: A Submodular Framework for Discovering Diverse Directions in GANs](https://mlsanity.com/paper/2203.08516/) (2022)
- [3DStyleGLIP: Part-Tailored Text-Guided 3D Neural Stylization](https://mlsanity.com/paper/2404.02634/) (2024)
- [LoRAverse: A Submodular Framework to Retrieve Diverse Adapters for Diffusion Models](https://mlsanity.com/paper/2510.15022/) (2025)
- [BiOcularGAN: Bimodal Synthesis and Annotation of Ocular Images](https://mlsanity.com/paper/2205.01536/) (2022)
- [Self-Supervised One-Shot Learning for Automatic Segmentation of StyleGAN Images](https://mlsanity.com/paper/2303.05639/) (2023)
- [Domain-Scalable Unpaired Image Translation via Latent Space Anchoring](https://mlsanity.com/paper/2306.14879/) (2023)
- [Contrastive Abstraction for Reinforcement Learning](https://mlsanity.com/paper/2410.00704/) (2024)
- [SketchYourSeg: Mask-Free Subjective Image Segmentation via Freehand Sketches](https://mlsanity.com/paper/2501.16022/) (2025)
- [TextCLIP: Text-Guided Face Image Generation And Manipulation Without Adversarial Training](https://mlsanity.com/paper/2309.11923/) (2023)
- [Linking in Style: Understanding learned features in deep learning models](https://mlsanity.com/paper/2409.16865/) (2024)

## Abstract (arXiv)

We introduce a method that allows to automatically segment images into semantically meaningful regions without human supervision. Derived regions are consistent across different images and coincide with human-defined semantic classes on some datasets. In cases where semantic regions might be hard for human to define and consistently label, our method is still able to find meaningful and consistent semantic classes. In our work, we use pretrained StyleGAN2 generative model: clustering in the feature space of the generative model allows to discover semantic classes. Once classes are discovered, a synthetic dataset with generated images and corresponding segmentation masks can be created. After that a segmentation model is trained on the synthetic dataset and is able to generalize to real images. Additionally, by using CLIP we are able to use prompts defined in a natural language to discover some desired semantic classes. We test our method on publicly available datasets and show state-of-the-art results.