mlsanityData to 7 Oct 2026

Unsupervised Discovery of Semantic Latent Directions in Diffusion Models

arXiv 2302.12469 ↗First posted 24 Feb 2023cs.CVcited by 28

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Despite the success of diffusion models (DMs), we still lack a thorough understanding of their latent space. While image editing with GANs builds upon latent space, DMs rely on editing the conditions such as text prompts. We present an unsupervised method to discover interpretable editing directions for the latent variables xt ∈ X of DMs. Our method adopts Riemannian geometry between X and the intermediate feature maps H of the U-Nets to provide a deep understanding over the geometrical structure of X. The discovered semantic latent directions mostly yield disentangled attribute changes, and they are globally consistent across different samples. Furthermore, editing in earlier timesteps edits coarse attributes, while ones in later timesteps focus on high-frequency details. We define the curvedness of a line segment between samples to show that X is a curved manifold. Experiments on different baselines and datasets demonstrate the effectiveness of our method even on Stable Diffusion. Our source code will be publicly available for the future researchers.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 42 of its 44 references that are in the map. First posted on arXiv 24 Feb 2023.

Builds on

42 of 44 references.

Closest ancestors · 13

Same direction, newest first

  1. 2022Diffusion Models already have a Semantic Latent Space20 Oct
  2. 2022Classifier-Free Diffusion Guidance26 Jul
  3. 2022gDDIM: Generalized denoising diffusion implicit models11 Jun
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  5. 2022Perception Prioritized Training of Diffusion Models1 Apr
  6. 2022Generating High Fidelity Data from Low-density Regions using Diffusion Models31 Mar
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  8. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models6 Aug
  9. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  10. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  11. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  12. 2020Denoising Diffusion Implicit Models6 Oct
1 more
  1. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2022Generative Adversarial Networkscited by 5,603
  6. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  7. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  8. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  9. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  10. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  11. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  12. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
17 more
  1. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  2. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  3. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  4. 2020Closed-Form Factorization of Latent Semantics in GANscited by 407
  5. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 348
  6. 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
  7. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidancecited by 211
  8. 2017Latent Space Oddity: on the Curvature of Deep Generative Modelscited by 169
  9. 2017The Riemannian Geometry of Deep Generative Modelscited by 81
  10. 2022MagicMix: Semantic Mixing with Diffusion Modelscited by 63
  11. 2017Metrics for Deep Generative Modelscited by 53
  12. 2021LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directionscited by 52
  13. 2021Low-Rank Subspaces in GANscited by 52
  14. 2020Geometrically Enriched Latent Spacescited by 45
  15. 2022Multiresolution Textual Inversioncited by 33
  16. 2018A Spectral Regularizer for Unsupervised Disentanglementcited by 30
  17. 2021Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANscited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 9; 2025: 6; 2026: 4.

28 papers.

Sort:

By area:

  1. 2023Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Modelscited by 148
  2. 2023A Neural Space-Time Representation for Text-to-Image Personalizationcited by 123
  3. 2023Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Modelscited by 60
  4. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  5. 2024Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversioncited by 58
  6. 2023Training-free Content Injection using h-space in Diffusion Modelscited by 36
  7. 2024Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Spacecited by 33
  8. 2023Latent Space Editing in Transformer-Based Flow Matchingcited by 32
  9. 2023ChatFace: Chat-Guided Real Face Editing via Diffusion Latent Space Manipulationcited by 16
  10. 2024Uncovering the Text Embedding in Text-to-Image Diffusion Modelscited by 13
  11. 2023Flow Factorized Representation Learningcited by 11
  12. 2023Unsupervised Discovery of Interpretable Directions in h-space of Pre-trained Diffusion Modelscited by 6
16 more
  1. 2025Interpreting Large Text-to-Image Diffusion Models with Dictionary Learningcited by 6
  2. 2024An Analysis of Human Alignment of Latent Diffusion Modelscited by 5
  3. 2025Image Interpolation with Score-based Riemannian Metrics of Diffusion Modelscited by 4
  4. 2024Restyling Unsupervised Concept Based Interpretable Networks with Generative Modelscited by 2
  5. 2025SCALEX: Scalable Concept and Latent Exploration for Diffusion Modelscited by 2
  6. 2025HMARK: Radioactive Multi-Bit Semantic-Latent Watermarking for Diffusion Modelscited by 2
  7. 2024Decoding Diffusion: A Scalable Framework for Unsupervised Analysis of Latent Space Biases and Representations Using Natural Language Promptscited by 1
  8. 2025Latent Diffusion U-Net Representations Contain Positional Embeddings and Anomaliescited by 1
  9. 2025Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Modelscited by 1
  10. 2024Understanding the Limitations of Diffusion Concept Algebra Through Foodcited by 0
  11. 2024Conjuring Semantic Similaritycited by 0
  12. 2024Unsupervised Panoptic Interpretation of Latent Spaces in GANs Using Space-Filling Vector Quantizationcited by 0
  13. 2026Learning Action Hierarchies via Hybrid Geometric Diffusioncited by 0
  14. 2026TokenDial: Continuous Attribute Control for Text-to-Video Generation in Visual Dial Spacecited by 0
  15. 2026Guidance for Low-Level Perceptual Editing in Unconditional Diffusion Modelscited by 0
  16. 2026From Noise to Control: Parameterized Diffusion Policiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enabling Local Editing in Diffusion Models by Joint and Individual Component Analysis
  2. 2023NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models
  3. 2023Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry
  4. 2024Unsupervised Region-Based Image Editing of Denoising Diffusion Models
  5. 2024Exploring Low-Dimensional Subspaces in Diffusion Models for Controllable Image Editing
  6. 2025Self-Attention Decomposition For Training Free Diffusion Editing
  7. 2026Geometric Decoupling: Diagnosing the Structural Instability of Latent
  8. 2024Isometric Representation Learning for Disentangled Latent Space of Diffusion Models