# Unsupervised Compositional Concepts Discovery with Text-to-Image Generative Models

arXiv: https://arxiv.org/abs/2306.05357 · HTML: https://mlsanity.com/paper/2306.05357/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 56 references that are in the map.
- First posted on arXiv 8 Jun 2023.

## Builds on (most cited first, 30 of 45)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [Object-Centric Learning with Slot Attention](https://mlsanity.com/paper/2006.15055/) (2020)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [MONet: Unsupervised Scene Decomposition and Representation](https://mlsanity.com/paper/1901.11390/) (2019)
- [Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis](https://mlsanity.com/paper/2212.05032/) (2022)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [Composer: Creative and Controllable Image Synthesis with Composable Conditions](https://mlsanity.com/paper/2302.09778/) (2023)
- [On the "steerability" of generative adversarial networks](https://mlsanity.com/paper/1907.07171/) (2019)
- [What the DAAM: Interpreting Stable Diffusion Using Cross Attention](https://mlsanity.com/paper/2210.04885/) (2022)

## Built on by (most cited first, 30 of 36)

- [Multi-LoRA Composition for Image Generation](https://mlsanity.com/paper/2402.16843/) (2024)
- [Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmenter](https://mlsanity.com/paper/2309.02773/) (2023)
- [NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models](https://mlsanity.com/paper/2312.05390/) (2023)
- [Compositional Generative Modeling: A Single Model is Not All You Need](https://mlsanity.com/paper/2402.01103/) (2024)
- [Interpretable Diffusion via Information Decomposition](https://mlsanity.com/paper/2310.07972/) (2023)
- [Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Space](https://mlsanity.com/paper/2406.19370/) (2024)
- [Energy-Based Concept Bottleneck Models: Unifying Prediction, Concept Intervention, and Probabilistic Interpretations](https://mlsanity.com/paper/2401.14142/) (2024)
- [Compositional Sculpting of Iterative Generative Processes](https://mlsanity.com/paper/2309.16115/) (2023)
- [SliderSpace: Decomposing the Visual Capabilities of Diffusion Models](https://mlsanity.com/paper/2502.01639/) (2025)
- [CoVLM: Composing Visual Entities and Relationships in Large Language Models Via Communicative Decoding](https://mlsanity.com/paper/2311.03354/) (2023)
- [Stylus: Automatic Adapter Selection for Diffusion Models](https://mlsanity.com/paper/2404.18928/) (2024)
- [Learning Discrete Concepts in Latent Hierarchical Models](https://mlsanity.com/paper/2406.00519/) (2024)
- [ConceptExpress: Harnessing Diffusion Models for Single-image Unsupervised Concept Extraction](https://mlsanity.com/paper/2407.07077/) (2024)
- [Language-Informed Visual Concept Learning](https://mlsanity.com/paper/2312.03587/) (2023)
- [Compositional Image Decomposition with Diffusion Models](https://mlsanity.com/paper/2406.19298/) (2024)
- [Compositional Video Generation as Flow Equalization](https://mlsanity.com/paper/2407.06182/) (2024)
- [Denoising Heat-inspired Diffusion with Insulators for Collision Free Motion Planning](https://mlsanity.com/paper/2310.12609/) (2023)
- [Learn Once, Edit Anywhere: Visual Direction Transfer for Diffusion Models](https://mlsanity.com/paper/2403.19645/) (2024)
- [Flexible Multitask Learning with Factorized Diffusion Policy](https://mlsanity.com/paper/2512.21898/) (2025)
- [Do Diffusion Models Learn Semantically Meaningful and Efficient Representations?](https://mlsanity.com/paper/2402.03305/) (2024)
- [Interpretable Measures of Conceptual Similarity by Complexity-Constrained Descriptive Auto-Encoding](https://mlsanity.com/paper/2402.08919/) (2024)
- [Few-Shot Task Learning through Inverse Generative Modeling](https://mlsanity.com/paper/2411.04987/) (2024)
- [Harnessing Caption Detailness for Data-Efficient Text-to-Image Generation](https://mlsanity.com/paper/2505.15172/) (2025)
- [Compositional Scene Understanding through Inverse Generative Modeling](https://mlsanity.com/paper/2505.21780/) (2025)
- [LoRAverse: A Submodular Framework to Retrieve Diverse Adapters for Diffusion Models](https://mlsanity.com/paper/2510.15022/) (2025)
- [ELROND: Exploring and decomposing intrinsic capabilities of diffusion models](https://mlsanity.com/paper/2602.10216/) (2026)
- [Enhancing Zero-Shot Image Recognition in Vision-Language Models through Human-like Concept Guidance](https://mlsanity.com/paper/2503.15886/) (2025)
- [DecompSR: A dataset for decomposed analyses of compositional multihop spatial reasoning](https://mlsanity.com/paper/2511.02627/) (2025)
- [SEDGE: Structural Extrapolated Data Generation](https://mlsanity.com/paper/2604.02482/) (2026)
- [Long-Text-to-Image Generation via Compositional Prompt Decomposition](https://mlsanity.com/paper/2604.18258/) (2026)

## Abstract (arXiv)

Text-to-image generative models have enabled high-resolution image synthesis across different domains, but require users to specify the content they wish to generate. In this paper, we consider the inverse problem -- given a collection of different images, can we discover the generative concepts that represent each image? We present an unsupervised approach to discover generative concepts from a collection of images, disentangling different art styles in paintings, objects, and lighting from kitchen scenes, and discovering image classes given ImageNet images. We show how such generative concepts can accurately represent the content of images, be recombined and composed to generate new artistic and hybrid images, and be further used as a representation for downstream classification tasks.