# Cluster-guided Image Synthesis with Unconditional Models

arXiv: https://arxiv.org/abs/2112.12911 · HTML: https://mlsanity.com/paper/2112.12911/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 46 references that are in the map.
- First posted on arXiv 24 Dec 2021.

## Builds on (most cited first, 30 of 39)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [Unsupervised Discovery of Interpretable Directions in the GAN Latent Space](https://mlsanity.com/paper/2002.03754/) (2020)
- [cGANs with Projection Discriminator](https://mlsanity.com/paper/1802.05637/) (2018)
- [On the "steerability" of generative adversarial networks](https://mlsanity.com/paper/1907.07171/) (2019)
- [Joint Unsupervised Learning of Deep Representations and Image Clusters](https://mlsanity.com/paper/1604.03628/) (2016)
- [Local Aggregation for Unsupervised Learning of Visual Embeddings](https://mlsanity.com/paper/1903.12355/) (2019)
- [Editing in Style: Uncovering the Local Semantics of GANs](https://mlsanity.com/paper/2004.14367/) (2020)
- [Visual Object Networks: Image Generation with Disentangled 3D Representation](https://mlsanity.com/paper/1812.02725/) (2018)
- [Controlling generative models with continuous factors of variations](https://mlsanity.com/paper/2001.10238/) (2020)
- [ClusterFit: Improving Generalization of Visual Representations](https://mlsanity.com/paper/1912.03330/) (2019)
- [DeLiGAN : Generative Adversarial Networks for Diverse and Limited Data](https://mlsanity.com/paper/1706.02071/) (2017)
- [ClusterGAN : Latent Space Clustering in Generative Adversarial Networks](https://mlsanity.com/paper/1809.03627/) (2018)
- [FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discovery](https://mlsanity.com/paper/1811.11155/) (2018)
- [Diverse Image Generation via Self-Conditioned GANs](https://mlsanity.com/paper/2006.10728/) (2020)
- [Implicit Maximum Likelihood Estimation](https://mlsanity.com/paper/1809.09087/) (2018)
- [High-Fidelity Synthesis with Disentangled Representation](https://mlsanity.com/paper/2001.04296/) (2020)

## Built on by (most cited first, 3 of 3)

- [Image Clustering via the Principle of Rate Reduction in the Age of Pretrained Models](https://mlsanity.com/paper/2306.05272/) (2023)
- [PandA: Unsupervised Learning of Parts and Appearances in the Feature Maps of GANs](https://mlsanity.com/paper/2206.00048/) (2022)
- [Discovering Intersectional Bias via Directional Alignment in Face Recognition Embeddings](https://mlsanity.com/paper/2510.15520/) (2025)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) are the driving force behind the state-of-the-art in image generation. Despite their ability to synthesize high-resolution photo-realistic images, generating content with on-demand conditioning of different granularity remains a challenge. This challenge is usually tackled by annotating massive datasets with the attributes of interest, a laborious task that is not always a viable option. Therefore, it is vital to introduce control into the generation process of unsupervised generative models. In this work, we focus on controllable image generation by leveraging GANs that are well-trained in an unsupervised fashion. To this end, we discover that the representation space of intermediate layers of the generator forms a number of clusters that separate the data according to semantically meaningful attributes (e.g., hair color and pose). By conditioning on the cluster assignments, the proposed method is able to control the semantic class of the generated image. Our approach enables sampling from each cluster by Implicit Maximum Likelihood Estimation (IMLE). We showcase the efficacy of our approach on faces (CelebA-HQ and FFHQ), animals (Imagenet) and objects (LSUN) using different pre-trained generative models. The results highlight the ability of our approach to condition image generation on attributes like gender, pose and hair style on faces, as well as a variety of features on different object classes.