# Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry

arXiv: https://arxiv.org/abs/2307.12868 · HTML: https://mlsanity.com/paper/2307.12868/

## Key facts

- Cited by 125 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 54 of its 62 references that are in the map.
- First posted on arXiv 24 Jul 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 54)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)

## Built on by (most cited first, 30 of 125)

- [Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models](https://mlsanity.com/paper/2311.12092/) (2023)
- [SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders](https://mlsanity.com/paper/2501.18052/) (2025)
- [Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation](https://mlsanity.com/paper/2311.17216/) (2023)
- [Smoothed Energy Guidance: Guiding Diffusion Models with Reduced Energy Curvature of Attention](https://mlsanity.com/paper/2408.00760/) (2024)
- [NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models](https://mlsanity.com/paper/2312.05390/) (2023)
- [Interpreting the Weight Space of Customized Diffusion Models](https://mlsanity.com/paper/2406.09413/) (2024)
- [Open-Vocabulary Attention Maps with Token Optimization for Semantic Segmentation in Diffusion Models](https://mlsanity.com/paper/2403.14291/) (2024)
- [Diamond Maps: Efficient Reward Alignment via Stochastic Flow Maps](https://mlsanity.com/paper/2602.05993/) (2026)
- [Isometric Representation Learning for Disentangled Latent Space of Diffusion Models](https://mlsanity.com/paper/2407.11451/) (2024)
- [Drag Your Gaussian: Effective Drag-Based Editing with Score Distillation for 3D Gaussian Splatting](https://mlsanity.com/paper/2501.18672/) (2025)
- [Emergence and Evolution of Interpretable Concepts in Diffusion Models](https://mlsanity.com/paper/2504.15473/) (2025)
- [Beta Sampling is All You Need: Efficient Image Generation Strategy for Diffusion Models using Stepwise Spectral Analysis](https://mlsanity.com/paper/2407.12173/) (2024)
- [Learning Discrete Concepts in Latent Hierarchical Models](https://mlsanity.com/paper/2406.00519/) (2024)
- [DiffusionPID: Interpreting Diffusion via Partial Information Decomposition](https://mlsanity.com/paper/2406.05191/) (2024)
- [TIDE : Temporal-Aware Sparse Autoencoders for Interpretable Diffusion Transformers in Image Generation](https://mlsanity.com/paper/2503.07050/) (2025)
- [Global Counterfactual Directions](https://mlsanity.com/paper/2404.12488/) (2024)
- [Exploring Low-Dimensional Subspaces in Diffusion Models for Controllable Image Editing](https://mlsanity.com/paper/2409.02374/) (2024)
- [Prompt Sliders for Fine-Grained Control, Editing and Erasing of Concepts in Diffusion Models](https://mlsanity.com/paper/2409.16535/) (2024)
- [Diffusion Counterfactual Generation with Semantic Abduction](https://mlsanity.com/paper/2506.07883/) (2025)
- [SAEmnesia: Erasing Concepts in Diffusion Models with Supervised Sparse Autoencoders](https://mlsanity.com/paper/2509.21379/) (2025)
- [Counterfactual Explanations via Riemannian Latent Space Traversal](https://mlsanity.com/paper/2411.02259/) (2024)
- [Hessian Geometry of Latent Space in Generative Models](https://mlsanity.com/paper/2506.10632/) (2025)
- [Bures-Wasserstein Flow Matching for Graph Generation](https://mlsanity.com/paper/2506.14020/) (2025)
- [Linear combinations of latents in generative models: subspaces and beyond](https://mlsanity.com/paper/2408.08558/) (2024)
- [HARIVO: Harnessing Text-to-Image Models for Video Generation](https://mlsanity.com/paper/2410.07763/) (2024)
- [Diffusion Generative Modeling on Lie Group Representations](https://mlsanity.com/paper/2502.02513/) (2025)
- [Responsible Diffusion Models via Constraining Text Embeddings within Safe Regions](https://mlsanity.com/paper/2505.15427/) (2025)
- [Unleashing the Potential of the Semantic Latent Space in Diffusion Models for Image Dehazing](https://mlsanity.com/paper/2509.20091/) (2025)
- [Learn Once, Edit Anywhere: Visual Direction Transfer for Diffusion Models](https://mlsanity.com/paper/2403.19645/) (2024)
- [Joint Diffusion models in Continual Learning](https://mlsanity.com/paper/2411.08224/) (2024)

## Abstract (arXiv)

Despite the success of diffusion models (DMs), we still lack a thorough understanding of their latent space. To understand the latent space xt ∈ X, we analyze them from a geometrical perspective. Our approach involves deriving the local latent basis within X by leveraging the pullback metric associated with their encoding feature maps. Remarkably, our discovered local latent basis enables image editing capabilities by moving xt, the latent space of DMs, along the basis vector at specific timesteps. We further analyze how the geometric structure of DMs evolves over diffusion timesteps and differs across different text conditions. This confirms the known phenomenon of coarse-to-fine generation, as well as reveals novel insights such as the discrepancy between xt across timesteps, the effect of dataset complexity, and the time-varying influence of text prompts. To the best of our knowledge, this paper is the first to present image editing through x-space traversal, editing only once at specific timestep t without any additional training, and providing thorough analyses of the latent structure of DMs. The code to reproduce our experiments can be found at https://github.com/enkeejunior1/Diffusion-Pullback.