mlsanityData to 14 Oct 2026

Loss Curvature as a Sampling-Free Proxy for Memorization in Diffusion Models

arXiv 2610.19042 ↗First posted 13 Oct 2026cs.LG cs.AI stat.MLcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Understanding the mechanism behind why diffusion models memorize their training data remains an open question in generative modeling. In this paper, we examine this question from two perspectives: (a) the model-level perspective concerning the onset of memorization in a model during training, and (b) the sample-level perspective concerning the proclivity of an individual sample to be memorized by a given model. We show that the so-called input loss curvature, or the Frobenius norm of the Hessian of the diffusion model's training loss, is an effective metric for quantifying both perspectives on memorization. In particular, we empirically demonstrate that input curvatures averaged across training and validation samples can identify when a model is likely to start regurgitating its training data when tracked over training time. However, this metric is not sufficient for measuring an individual sample's likelihood of being memorized. To address this perspective, we introduce a counterfactual definition of memorization in diffusion models, extending the analogous notion from supervised learning proposed by [Feldman, 2019]. We prove that this counterfactual memorization is controlled by the input curvature of the leave-one-out model, thus establishing a direct theoretical link between a sample's local loss landscape and its generation likelihood. We validate our derived bounds through several empirical experiments and show that input-dependent curvature provides a proxy for memorization.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 40 of its 53 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

40 of 53 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training23 May
  2. 2025Memorization and Regularization in Generative Diffusion Models27 Jan
  3. 2024Losing dimensions: Geometric memorization in generative diffusion11 Oct
  4. 2024Dynamical Regimes of Diffusion Models28 Feb
  5. 2023On Memorization in Diffusion Models4 Oct
  6. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
  7. 2023Understanding and Mitigating Copying in Diffusion Models31 May
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  9. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matching5 Jun
  10. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  11. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  12. 2020Denoising Diffusion Implicit Models6 Oct
1 more
  1. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2022Flow Matching for Generative Modelingcited by 5,702
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  9. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  10. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  11. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  12. 2020Extracting Training Data from Large Language Modelscited by 2,189
15 more
  1. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  2. 2021Membership Inference Attacks From First Principlescited by 878
  3. 2023Extracting Training Data from Diffusion Modelscited by 732
  4. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 497
  5. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 460
  6. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 424
  7. 2020Loss landscapes and optimization in over-parameterized non-linear systems and neural networkscited by 233
  8. 2018A mathematical theory of semantic development in deep neural networkscited by 186
  9. 2018Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balancedcited by 177
  10. 2021Counterfactual Memorization in Neural Language Modelscited by 156
  11. 2021On Linear Stability of SGD and Input-Smoothness of Neural Networkscited by 63
  12. 2023Memorization Through the Lens of Curvature of Loss Function Around Samplescited by 27
  13. 2024Unveiling Privacy, Memorization, and Input Curvature Linkscited by 10
  14. 2024Transformer Block Coupling and its Correlation with Generalization in LLMscited by 10
  15. 2024Curvature Clues: Decoding Deep Learning Privacy with Input Loss Curvaturecited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025On the Edge of Memorization in Diffusion Models
  2. 2026Localizing Memorized Regions in Diffusion Models via Coordinate-Wise Curvature Differences
  3. 2026Early Signatures of Memorization in Diffusion Models via Basin Geometry and Cyclic Denoising
  4. 2025How Diffusion Models Memorize
  5. 2026Local Coverage Governs Memorization in Diffusion Models
  6. 2024Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes
  7. 2026Memorization Control in Diffusion Models from Denoising-centric Perspective
  8. 2026Two Calm Ends and the Wild Middle: A Geometric Picture of Memorization in Diffusion Models