# Compositional Inversion for Stable Diffusion Models arXiv: https://arxiv.org/abs/2312.08048 ยท HTML: https://mlsanity.com/paper/2312.08048/ ## Key facts - Cited by 19 papers in the map (data through 14 Oct 2026). - Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. - Builds on 22 of its 31 references that are in the map. - First posted on arXiv 13 Dec 2023. ## Placed in - Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/) ## Builds on (most cited first, 22 of 22) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018) - [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023) - [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021) - [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022) - [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022) - [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022) - [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022) - [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021) - [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022) - [SVDiff: Compact Parameter Space for Diffusion Fine-Tuning](https://mlsanity.com/paper/2303.11305/) (2023) - [Training-Free Layout Control with Cross-Attention Guidance](https://mlsanity.com/paper/2304.03373/) (2023) - [DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis](https://mlsanity.com/paper/1904.01310/) (2019) - [Key-Locked Rank One Editing for Text-to-Image Personalization](https://mlsanity.com/paper/2305.01644/) (2023) - [Cones: Concept Neurons in Diffusion Models for Customized Generation](https://mlsanity.com/paper/2303.05125/) (2023) ## Built on by (most cited first, 19 of 19) - [Controllable Generation with Text-to-Image Diffusion Models: A Survey](https://mlsanity.com/paper/2403.04279/) (2024) - [OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models](https://mlsanity.com/paper/2403.10983/) (2024) - [A Survey on Personalized Content Synthesis with Diffusion Models](https://mlsanity.com/paper/2405.05538/) (2024) - [MC2: Multi-concept Guidance for Customized Multi-concept Generation](https://mlsanity.com/paper/2404.05268/) (2024) - [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025) - [OneActor: Consistent Character Generation via Cluster-Conditioned Guidance](https://mlsanity.com/paper/2404.10267/) (2024) - [CoRe: Context-Regularized Text Embedding Learning for Text-to-Image Personalization](https://mlsanity.com/paper/2408.15914/) (2024) - [CSD-VAR: Content-Style Decomposition in Visual Autoregressive Models](https://mlsanity.com/paper/2507.13984/) (2025) - [Generative Active Learning for Image Synthesis Personalization](https://mlsanity.com/paper/2403.14987/) (2024) - [Defect Image Sample Generation With Diffusion Prior for Steel Surface Defect Recognition](https://mlsanity.com/paper/2405.01872/) (2024) - [GD-Retriever: Controllable Generative Text-Music Retrieval with Diffusion Models](https://mlsanity.com/paper/2506.17886/) (2025) - [ConceptSplit: Decoupled Multi-Concept Personalization of Diffusion Models via Token-wise Adaptation and Attention Disentanglement](https://mlsanity.com/paper/2510.04668/) (2025) - [MasterWeaver: Taming Editability and Face Identity for Personalized Text-to-Image Generation](https://mlsanity.com/paper/2405.05806/) (2024) - [When and Where do Data Poisons Attack Textual Inversion?](https://mlsanity.com/paper/2507.10578/) (2025) - [Directional Textual Inversion for Personalized Text-to-Image Generation](https://mlsanity.com/paper/2512.13672/) (2025) - [PersonaCraft: Personalized and Controllable Full-Body Multi-Human Scene Generation Using Occlusion-Aware 3D-Conditioned Diffusion](https://mlsanity.com/paper/2411.18068/) (2024) - [VMDiff: Visual Mixing Diffusion for Limitless Cross-Object Synthesis](https://mlsanity.com/paper/2509.23605/) (2025) - [Which Way from B to A: The role of embedding geometry in image interpolation for Stable Diffusion](https://mlsanity.com/paper/2511.12757/) (2025) - [From Competition to Coopetition: Coopetitive Training-Free Image Editing Based on Text Guidance](https://mlsanity.com/paper/2604.15948/) (2026) ## Abstract (arXiv) Inversion methods, such as Textual Inversion, generate personalized images by incorporating concepts of interest provided by user images. However, existing methods often suffer from overfitting issues, where the dominant presence of inverted concepts leads to the absence of other desired concepts. It stems from the fact that during inversion, the irrelevant semantics in the user images are also encoded, forcing the inverted concepts to occupy locations far from the core distribution in the embedding space. To address this issue, we propose a method that guides the inversion process towards the core distribution for compositional embeddings. Additionally, we introduce a spatial regularization approach to balance the attention on the concepts being composed. Our method is designed as a post-training approach and can be seamlessly integrated with other inversion methods. Experimental results demonstrate the effectiveness of our proposed approach in mitigating the overfitting problem and generating more diverse and balanced compositions of concepts in the synthesized images. The source code is available at https://github.com/zhangxulu1996/Compositional-Inversion.