# Hessian Geometry of Latent Space in Generative Models

arXiv: https://arxiv.org/abs/2506.10632 · HTML: https://mlsanity.com/paper/2506.10632/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 14 of 14)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [U2-Net: Going Deeper with Nested U-Structure for Salient Object Detection](https://mlsanity.com/paper/2005.09007/) (2020)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [Dynamical Regimes of Diffusion Models](https://mlsanity.com/paper/2402.18491/) (2024)
- [Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry](https://mlsanity.com/paper/2307.12868/) (2023)
- [A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data](https://mlsanity.com/paper/2402.16991/) (2024)
- [Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation](https://mlsanity.com/paper/2311.17216/) (2023)
- [Verifying the Union of Manifolds Hypothesis for Image Data](https://mlsanity.com/paper/2207.02862/) (2022)
- [Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models](https://mlsanity.com/paper/2312.04410/) (2023)
- [Learning Summary Statistic for Approximate Bayesian Computation via Deep Neural Network](https://mlsanity.com/paper/1510.02175/) (2015)
- [Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/2106.09016/) (2021)
- [Lipschitz Singularities in Diffusion Models](https://mlsanity.com/paper/2306.11251/) (2023)
- [Unveiling phase transitions with machine learning](https://mlsanity.com/paper/1904.01486/) (2019)

## Built on by (most cited first, 9 of 9)

- [Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Models](https://mlsanity.com/paper/2510.05509/) (2025)
- [Conditioned Activation Transport for T2I Safety Steering](https://mlsanity.com/paper/2603.03163/) (2026)
- [Geodesic Calculus on Implicitly Defined Latent Manifolds](https://mlsanity.com/paper/2510.09468/) (2025)
- [Rethinking Input Domains in Physics-Informed Neural Networks via Geometric Compactification Mappings](https://mlsanity.com/paper/2602.16193/) (2026)
- [Complex variational autoencoders admit K"ahler structure](https://mlsanity.com/paper/2511.15172/) (2025)
- [ImageRAGTurbo: Towards One-step Text-to-Image Generation with Retrieval-Augmented Diffusion Models](https://mlsanity.com/paper/2602.12640/) (2026)
- [Understanding Latent Diffusability via Fisher Geometry](https://mlsanity.com/paper/2604.02751/) (2026)
- [BRACE: Taming Sharp Irregularities via Barycentric Rational Forecasting for Fast Diffusion Transformers Inference](https://mlsanity.com/paper/2608.07572/) (2026)
- [The Loss Floor of Denoising Score Matching: Fisher Geometry from Schr"odinger Bridges](https://mlsanity.com/paper/2608.23916/) (2026)

## Abstract (arXiv)

This paper presents a novel method for analyzing the latent space geometry of generative models, including statistical physics models and diffusion models, by reconstructing the Fisher information metric. The method approximates the posterior distribution of latent variables given generated samples and uses this to learn the log-partition function, which defines the Fisher metric for exponential families. Theoretical convergence guarantees are provided, and the method is validated on the Ising and TASEP models, outperforming existing baselines in reconstructing thermodynamic quantities. Applied to diffusion models, the method reveals a fractal structure of phase transitions in the latent space, characterized by abrupt changes in the Fisher metric. We demonstrate that while geodesic interpolations are approximately linear within individual phases, this linearity breaks down at phase boundaries, where the diffusion model exhibits a divergent Lipschitz constant with respect to the latent space. These findings provide new insights into the complex structure of diffusion model latent spaces and their connection to phenomena like phase transitions. Our source code is available at https://github.com/alobashev/hessian-geometry-of-diffusion-models.