# Optimal Stopping in Latent Diffusion Models

arXiv: https://arxiv.org/abs/2510.08409 · HTML: https://mlsanity.com/paper/2510.08409/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 9 Oct 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [Rethinking FID: Towards a Better Evaluation Metric for Image Generation](https://mlsanity.com/paper/2401.09603/) (2023)
- [Riemannian Score-Based Generative Modelling](https://mlsanity.com/paper/2202.02763/) (2022)
- [Riemannian Diffusion Models](https://mlsanity.com/paper/2208.07949/) (2022)
- [EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modeling](https://mlsanity.com/paper/2502.09509/) (2025)
- [Improving the Diffusability of Autoencoders](https://mlsanity.com/paper/2502.14831/) (2025)
- [Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions](https://mlsanity.com/paper/2409.18804/) (2024)
- [Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models](https://mlsanity.com/paper/2405.14861/) (2024)
- [From Principal Subspaces to Principal Components with Linear Autoencoders](https://mlsanity.com/paper/1804.10253/) (2018)
- [Lipschitz Singularities in Diffusion Models](https://mlsanity.com/paper/2306.11251/) (2023)
- [Mathematical analysis of singularities in the diffusion model under the submanifold assumption](https://mlsanity.com/paper/2301.07882/) (2023)
- [Diffusion models for Gaussian distributions: Exact solutions and Wasserstein errors](https://mlsanity.com/paper/2405.14250/) (2024)
- [Alias-Free Latent Diffusion Models: Improving Fractional Shift Equivariance of Diffusion Latent Space](https://mlsanity.com/paper/2503.09419/) (2025)
- [Analysis of Diffusion Models for Manifold Data](https://mlsanity.com/paper/2502.04339/) (2025)
- [From Score Matching to Diffusion: A Fine-Grained Error Analysis in the Gaussian Setting](https://mlsanity.com/paper/2503.11615/) (2025)

## Built on by (most cited first, 2 of 2)

- [A Gaussian Perspective for Distributional Discrepancy in Generative Diffusion Models](https://mlsanity.com/paper/2601.13602/) (2026)
- [MAGNET: Manifold-Aware Graph Diffusion Network for Connectome Generation](https://mlsanity.com/paper/2610.17014/) (2026)

## Abstract (arXiv)

We identify and analyze a surprising phenomenon of Latent Diffusion Models (LDMs) where the final steps of the diffusion can degrade sample quality. In contrast to conventional arguments that justify early stopping for numerical stability, this phenomenon is intrinsic to the dimensionality reduction in LDMs. We provide a principled explanation by analyzing the interaction between latent dimension and stopping time. Under a Gaussian framework with linear autoencoders, we characterize the conditions under which early stopping is needed to minimize the distance between generated and target distributions. More precisely, we show that lower-dimensional representations benefit from earlier termination, whereas higher-dimensional latent spaces require later stopping time. We further establish that the latent dimension interplays with other hyperparameters of the problem such as constraints in the parameters of score matching. Experiments on synthetic and real datasets illustrate these properties, underlining that early stopping can improve generative quality. Together, our results offer a theoretical foundation for understanding how the latent dimension influences the sample quality, and highlight stopping time as a key hyperparameter in LDMs.