Cluster-guided Image Synthesis with Unconditional Models
Not on the map. Why?
Abstract (arXiv)
Generative Adversarial Networks (GANs) are the driving force behind the state-of-the-art in image generation. Despite their ability to synthesize high-resolution photo-realistic images, generating content with on-demand conditioning of different granularity remains a challenge. This challenge is usually tackled by annotating massive datasets with the attributes of interest, a laborious task that is not always a viable option. Therefore, it is vital to introduce control into the generation process of unsupervised generative models. In this work, we focus on controllable image generation by leveraging GANs that are well-trained in an unsupervised fashion. To this end, we discover that the representation space of intermediate layers of the generator forms a number of clusters that separate the data according to semantically meaningful attributes (e.g., hair color and pose). By conditioning on the cluster assignments, the proposed method is able to control the semantic class of the generated image. Our approach enables sampling from each cluster by Implicit Maximum Likelihood Estimation (IMLE). We showcase the efficacy of our approach on faces (CelebA-HQ and FFHQ), animals (Imagenet) and objects (LSUN) using different pre-trained generative models. The results highlight the ability of our approach to condition image generation on attributes like gender, pose and hair style on faces, as well as a variety of features on different object classes.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 39 of its 46 references that are in the map. First posted on arXiv 24 Dec 2021.
Builds on
39 of 46 references.
References · 39
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
- 2017Billion-scale similarity search with GPUscited by 2,477
- 2016Generative Adversarial Imitation Learningcited by 1,576
- 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
- 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
- 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
- 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
27 more
- 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
- 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
- 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
- 2020Unsupervised Discovery of Interpretable Directions in the GAN Latent Spacecited by 288
- 2018cGANs with Projection Discriminatorcited by 253
- 2019On the "steerability" of generative adversarial networkscited by 247
- 2016Joint Unsupervised Learning of Deep Representations and Image Clusterscited by 205
- 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
- 2020Editing in Style: Uncovering the Local Semantics of GANscited by 183
- 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
- 2020Controlling generative models with continuous factors of variationscited by 93
- 2019ClusterFit: Improving Generalization of Visual Representationscited by 82
- 2017DeLiGAN : Generative Adversarial Networks for Diverse and Limited Datacited by 77
- 2018ClusterGAN : Latent Space Clustering in Generative Adversarial Networkscited by 75
- 2018FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discoverycited by 58
- 2020Diverse Image Generation via Self-Conditioned GANscited by 54
- 2018Implicit Maximum Likelihood Estimationcited by 52
- 2020High-Fidelity Synthesis with Disentangled Representationcited by 23
- 2019OOGAN: Disentangling GAN with One-Hot Sampling and Orthogonal Regularizationcited by 17
- 2020Multilinear Latent Conditioning for Generating Unseen Attribute Combinationscited by 14
- 2021Tensor Component Analysis for Interpreting the Latent Space of GANscited by 13
- 2021CoPE: Conditional image generation using Polynomial Expansionscited by 12
- 2019A Prior of a Googol Gaussians: a Tensor Ring Induced Prior for Generative Modelscited by 9
- 2019Yes, we GAN: Applying Adversarial Techniques for Autonomous Drivingcited by 7
- 2019Non-Parametric Priors For Generative Adversarial Networkscited by 5
- 2018Generative Adversarial Image Synthesis with Decision Tree Latent Controllercited by 3
- 2018Unsupervised Disentangled Representation Learning with Analogical Relationscited by 2
Built on by
Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 1.
3 papers.