# Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes

arXiv: https://arxiv.org/abs/2412.04140 · HTML: https://mlsanity.com/paper/2412.04140/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 36 of its 49 references that are in the map.
- First posted on arXiv 5 Dec 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 36)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [The Intrinsic Dimension of Images and Its Impact on Learning](https://mlsanity.com/paper/2104.08894/) (2021)
- [Understanding and Mitigating Copying in Diffusion Models](https://mlsanity.com/paper/2305.20086/) (2023)
- [A Self-Supervised Descriptor for Image Copy Detection](https://mlsanity.com/paper/2202.10261/) (2022)
- [On Memorization in Diffusion Models](https://mlsanity.com/paper/2310.02664/) (2023)
- [Score-Based Generative Models Detect Manifolds](https://mlsanity.com/paper/2206.01018/) (2022)
- [Generating Long Videos of Dynamic Scenes](https://mlsanity.com/paper/2206.03429/) (2022)
- [Detecting, Explaining, and Mitigating Memorization in Diffusion Models](https://mlsanity.com/paper/2407.21720/) (2024)
- [Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matching](https://mlsanity.com/paper/2206.08265/) (2022)
- [Estimating High Order Gradients of the Data Distribution by Denoising](https://mlsanity.com/paper/2111.04726/) (2021)
- [A Reproducible Extraction of Training Images from Diffusion Models](https://mlsanity.com/paper/2305.08694/) (2023)
- [Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention](https://mlsanity.com/paper/2403.11052/) (2024)
- [A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Models](https://mlsanity.com/paper/2406.03537/) (2024)
- [A Geometric Framework for Understanding Memorization in Generative Models](https://mlsanity.com/paper/2411.00113/) (2024)
- [Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion](https://mlsanity.com/paper/2410.05898/) (2024)

## Built on by (most cited first, 21 of 21)

- [Making Reconstruction FID Predictive of Diffusion Generation FID](https://mlsanity.com/paper/2603.05630/) (2026)
- [Generalization of Diffusion Models Arises with a Balanced Representation Space](https://mlsanity.com/paper/2512.20963/) (2025)
- [How Diffusion Models Memorize](https://mlsanity.com/paper/2509.25705/) (2025)
- [Substitution-Based Analysis of Structural Novelty for Generative Models of Materials](https://mlsanity.com/paper/2606.23166/) (2026)
- [Not Every Time and Frequency Need to Be Forgotten in Diffusion Unlearning](https://mlsanity.com/paper/2510.17917/) (2025)
- [Transport Novelty Distance: A Distributional Metric for Evaluating Material Generative Models](https://mlsanity.com/paper/2512.09514/) (2025)
- [Detecting and Mitigating Memorization in Diffusion Models through Anisotropy of the Log-Probability](https://mlsanity.com/paper/2601.20642/) (2026)
- [Dominant vs. Dominated: Concept-Level Generative Collapse in Diffusion Models](https://mlsanity.com/paper/2512.20666/) (2025)
- [Local Hessian Spectral Filtering for Robust Intrinsic Dimension Estimation](https://mlsanity.com/paper/2605.01221/) (2026)
- [Initialization is Half the Battle: Generating Diverse Images from a Guidance Potential Posterior](https://mlsanity.com/paper/2606.02453/) (2026)
- [Identifying Memorization of Diffusion Models through p-Laplace Analysis: Estimators, Bounds and Applications](https://mlsanity.com/paper/2505.08246/) (2025)
- [You Don't Need All That Attention: Surgical Memorization Mitigation in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2603.00133/) (2026)
- [Memorization In Stable Diffusion Is Unexpectedly Driven by CLIP Embeddings](https://mlsanity.com/paper/2605.02908/) (2026)
- [Global Convergence of Sampling-Based Nonconvex Optimization through Diffusion-Style Smoothing](https://mlsanity.com/paper/2605.16520/) (2026)
- [Machine Unlearning for Masked Diffusion Language Models](https://mlsanity.com/paper/2605.18253/) (2026)
- [Broken Memories: Detecting and Mitigating Memorization in Diffusion Models with Degraded Generations](https://mlsanity.com/paper/2605.22050/) (2026)
- [Localizing Memorized Regions in Diffusion Models via Coordinate-Wise Curvature Differences](https://mlsanity.com/paper/2605.26756/) (2026)
- [Semi-Supervised Conditional Diffusion via Label Augmentation](https://mlsanity.com/paper/2607.16685/) (2026)
- [Fingerprinting Text-to-Image Diffusion Models via Collapsed Generation](https://mlsanity.com/paper/2608.11732/) (2026)
- [Background Gradients Shape Memorization in Flow Matching](https://mlsanity.com/paper/2609.33210/) (2026)
- [From Modes to Memories: Characterizing the Scale-Space Dynamics of Diffusion Models](https://mlsanity.com/paper/2609.39648/) (2026)

## Abstract (arXiv)

In this paper, we introduce a geometric framework to analyze memorization in diffusion models through the sharpness of the log probability density. We mathematically justify a previously proposed score-difference-based memorization metric by demonstrating its effectiveness in quantifying sharpness. Additionally, we propose a novel memorization metric that captures sharpness at the initial stage of image generation in latent diffusion models, offering early insights into potential memorization. Leveraging this metric, we develop a mitigation strategy that optimizes the initial noise of the generation process using a sharpness-aware regularization term. The code is publicly available at https://github.com/Dongjae0324/sharpness_memorization_diffusion.