mlsanityData to 7 Oct 2026

A Theoretical Analysis of Memory and Overfitting Phenomena in Stochastic Interpolation Models

arXiv 2606.08554 ↗First posted 7 Jun 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper provides a theoretical account of memorization in stochastic interpolation models. By leveraging closed-form expressions for the optimal velocity field and the associated score function, we show that, in the continuous-time oracle setting, both deterministic and stochastic generation processes recover training samples. Under Euler discretization, generated samples remain centered around training samples, with deviations controlled by the step size. We further analyze generation in the presence of estimation errors and show that accumulated estimation errors control the endpoint deviation from the training set. These results imply that the generated sample admits a representation as a training sample perturbed by three controlled terms: a discretization-induced bound, an estimation-error-induced bound, and stochastic Gaussian noise. Based on this characterization, we provide theoretical definitions of overfitting and underfitting in generative models. Synthetic simulations support our theoretical findings.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 7 Jun 2026.

Builds on

18 of 28 references.

References · 18

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
  4. 2023Understanding and Mitigating Copying in Diffusion Modelscited by 205
  5. 2025Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Trainingcited by 78
  6. 2023On the Generalization Properties of Diffusion Modelscited by 67
  7. 2024A Good Score Does not Lead to A Good Generative Modelcited by 36
  8. 2025On the Edge of Memorization in Diffusion Modelscited by 22
  9. 2024How Do Flow Matching Models Memorize and Generalize in Sample Data Subspaces?cited by 19
  10. 2025A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspectivecited by 18
  11. 2025Provable Separations between Memorization and Generalization in Diffusion Modelscited by 14
  12. 2025From Navigation to Refinement: Revealing the Two-Stage Nature of Flow-based Diffusion Models through Oracle Velocitycited by 7
6 more
  1. 2026Smoothing the Score Function to Enhance Generalization in Diffusion Modelscited by 3
  2. 2026Two Calm Ends and the Wild Middle: A Geometric Picture of Memorization in Diffusion Modelscited by 3
  3. 2025A Novel Metric for Detecting Memorization in Generative Models for Brain MRI Synthesiscited by 2
  4. 2026Diffusion Models Memorize in Training -- and Generalize in Inferencecited by 2
  5. 2026Manifold Generalization Provably Proceeds Memorization in Diffusion Modelscited by 2
  6. 2026Memorization Control in Diffusion Models from Denoising-centric Perspectivecited by 1

Built on by

Not cited yet.