# Resolving Memorization in Empirical Diffusion Model for Manifold Data in High-Dimensional Spaces

arXiv: https://arxiv.org/abs/2505.02508 · HTML: https://mlsanity.com/paper/2505.02508/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 5 May 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 18 of 18)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022)
- [Diffusion models as plug-and-play priors](https://mlsanity.com/paper/2206.09012/) (2022)
- [Diffusion Models are Minimax Optimal Distribution Estimators](https://mlsanity.com/paper/2303.01861/) (2023)
- [Score-Based Generative Models Detect Manifolds](https://mlsanity.com/paper/2206.01018/) (2022)
- [An analytic theory of creativity in convolutional diffusion models](https://mlsanity.com/paper/2412.20292/) (2024)
- [Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions](https://mlsanity.com/paper/2409.18804/) (2024)
- [Closed-Form Diffusion Models](https://mlsanity.com/paper/2310.12395/) (2023)
- [A Geometric Framework for Understanding Memorization in Generative Models](https://mlsanity.com/paper/2411.00113/) (2024)
- [Memorization and Regularization in Generative Diffusion Models](https://mlsanity.com/paper/2501.15785/) (2025)
- [A Good Score Does not Lead to A Good Generative Model](https://mlsanity.com/paper/2401.04856/) (2024)
- [The Emergence of Reproducibility and Generalizability in Diffusion Models](https://mlsanity.com/paper/2310.05264/) (2023)
- [Mathematical analysis of singularities in the diffusion model under the submanifold assumption](https://mlsanity.com/paper/2301.07882/) (2023)
- [Wasserstein proximal operators describe score-based generative models and resolve memorization](https://mlsanity.com/paper/2402.06162/) (2024)
- [Minimax adaptive estimation in manifold inference](https://mlsanity.com/paper/2001.04896/) (2020)
- [Permutation Recovery on Manifold Data via Spectral Seriation](https://mlsanity.com/paper/2406.13635/) (2024)

## Built on by (most cited first, 8 of 8)

- [When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis](https://mlsanity.com/paper/2509.24912/) (2025)
- [Generalization of Diffusion Models Arises with a Balanced Representation Space](https://mlsanity.com/paper/2512.20963/) (2025)
- [A PDE Perspective on Generative Diffusion Models](https://mlsanity.com/paper/2511.05940/) (2025)
- [On The Hidden Biases of Flow Matching Samplers](https://mlsanity.com/paper/2512.16768/) (2025)
- [Understanding diffusion models requires rethinking (again) generalization](https://mlsanity.com/paper/2605.06077/) (2026)
- [Intrinsic Wasserstein Rates for Score-Based Generative Models on Smooth Manifolds](https://mlsanity.com/paper/2605.15822/) (2026)
- [A Quantitative Approximation Framework for Flow Distillation in Diffusion Models](https://mlsanity.com/paper/2606.03820/) (2026)
- [Optimal score function estimation via derivatives constraints](https://mlsanity.com/paper/2606.19084/) (2026)

## Abstract (arXiv)

Diffusion models are popular tools for generating new data samples, using a forward process that adds noise to data and a reverse process to denoise and produce samples. However, when the data distribution consists of n points, empirical diffusion models tend to reproduce existing data points, a phenomenon known as the memorization effect. Current literature often addresses this with complex machine learning techniques. This work shows that the memorization issue can be solved simply by applying an inertia update at the end of the empirical diffusion simulation. Our inertial diffusion model requires only the empirical score function and no additional training. We demonstrate that the distribution of samples from this model approximates the true data distribution on a C2 manifold of dimension d, within a Wasserstein-1 distance of order O(n^-frac2d+4). This bound significantly shrinks the Wasserstein distance between the population and empirical distributions, confirming that the inertial diffusion model produces new and diverse samples. Remarkably, this estimate is independent of the ambient space dimension, as no further training is needed. Our analysis shows that the inertial diffusion samples resemble Gaussian kernel density estimations on the manifold, revealing a novel connection between diffusion models and manifold learning.