mlsanityData to 7 Oct 2026

Learning Mixtures of Gaussians Using the DDPM Objective

arXiv 2307.01178 ↗First posted 3 Jul 2023cs.DS cs.LG stat.MLcited by 56

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Recent works have shown that diffusion models can learn essentially any distribution provided one can perform score estimation. Yet it remains poorly understood under what settings score estimation is possible, let alone when practical gradient-based algorithms for this task can provably succeed. In this work, we give the first provably efficient results along these lines for one of the most fundamental distribution families, Gaussian mixture models. We prove that gradient descent on the denoising diffusion probabilistic model (DDPM) objective can efficiently recover the ground truth parameters of the mixture model in the following two settings: 1) We show gradient descent with random initialization learns mixtures of two spherical Gaussians in d dimensions with 1/poly(d)-separated centers. 2) We show gradient descent with a warm start learns mixtures of K spherical Gaussians with Omega(√log(min(K,d)))-separated centers. A key ingredient in our proofs is a new connection between score-based methods and two other approaches to distribution learning, the EM algorithm and spectral methods.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 40 of its 66 references that are in the map. First posted on arXiv 3 Jul 2023.

Builds on

40 of 66 references.

Closest ancestors · 15

Same direction, newest first

  1. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models15 Jun
  2. 2023Error Bounds for Flow Matching Methods26 May
  3. 2023The probability flow ODE is provably fast19 May
  4. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers6 Mar
  5. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  6. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  7. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  8. 2022Let us Build Bridges: Understanding and Extending Diffusion Generative Models31 Aug
  9. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  10. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  11. 2022Score-Based Generative Models Detect Manifolds2 Jun
  12. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
3 more
  1. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  3. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 25

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2016Variational Inference: A Review for Statisticianscited by 1,466
  6. 2014Statistical guarantees for the EM algorithm: From population to sample-based analysiscited by 131
  7. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 118
  8. 2016Statistical Query Lower Bounds for Robust Estimation of High-dimensional Gaussians and Gaussian Mixturescited by 114
  9. 2017Mixture Models, Robustness, and Sum of Squares Proofscited by 81
  10. 2017List-Decodable Robust Mean Estimation and Learning Mixtures of Spherical Gaussianscited by 70
  11. 2022Sampling from the Sherrington-Kirkpatrick Gibbs measure via algorithmic stochastic localizationcited by 48
  12. 2017Estimation of Low-Rank Matrices via Approximate Message Passingcited by 41
13 more
  1. 2016Ten Steps of EM Suffice for Mixtures of Two Gaussianscited by 40
  2. 2020Continuous LWEcited by 37
  3. 2016Global analysis of Expectation Maximization for mixtures of two Gaussianscited by 31
  4. 2017On Learning Mixtures of Well-Separated Gaussianscited by 30
  5. 2014Tight bounds for learning a mixture of two gaussianscited by 29
  6. 2020Robustly Learning any Clusterable Mixture of Gaussianscited by 28
  7. 2020Small Covers for Near-Zero Sets of Polynomials and Learning Latent Variable Modelscited by 28
  8. 2021Clustering Mixtures with Almost Optimal Separation in Polynomial Timecited by 28
  9. 2014Near-optimal-sample estimators for spherical Gaussian mixturescited by 26
  10. 2020Robust Learning of Mixtures of Gaussianscited by 15
  11. 2021Local convexity of the TAP free energy and AMP convergence for Z2-synchronizationcited by 15
  12. 2021Improved Convergence Guarantees for Learning Gaussian Mixture Models by EM and Gradient EMcited by 11
  13. 2022Sudakov-Fernique post-AMP, and a new proof of the local convexity of the TAP free energycited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 24; 2025: 17; 2026: 12.

56 papers.

Sort:

By area:

  1. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Modelscited by 93
  2. 2024A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Datacited by 88
  3. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  4. 2023On the Generalization Properties of Diffusion Modelscited by 67
  5. 2024Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theorycited by 58
  6. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributionscited by 53
  7. 2024What does guidance do? A fine-grained analysis in a simple settingcited by 45
  8. 2024Critical windows: non-asymptotic theory for feature emergence in diffusion modelscited by 41
  9. 2024Learning Mixtures of Gaussians Using Diffusion Modelscited by 41
  10. 2023Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Modelscited by 39
  11. 2024Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalizationcited by 39
  12. 2023Analysis of learning a flow-based generative model from limited sample complexitycited by 38
38 more
  1. 2024Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structurecited by 38
  2. 2024Evaluating the design space of diffusion-based generative modelscited by 25
  3. 2024Unraveling the Smoothness Properties of Diffusion Models: A Gaussian Mixture Perspectivecited by 24
  4. 2025How Compositional Generalization and Creativity Improve as Diffusion Models are Trainedcited by 24
  5. 2025Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Modelscited by 24
  6. 2025Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixturescited by 23
  7. 2025On the Edge of Memorization in Diffusion Modelscited by 22
  8. 2025Denoising Score Matching with Random Features: Insights on Diffusion Models from Precise Learning Curvescited by 20
  9. 2025Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptivecited by 20
  10. 2024How much is a noisy image worth? Data Scaling Laws for Ambient Diffusioncited by 18
  11. 2024U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Modelscited by 17
  12. 2025Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusioncited by 17
  13. 2024Diffusion models for Gaussian distributions: Exact solutions and Wasserstein errorscited by 15
  14. 2025On the Interpolation Effect of Score Smoothing in Diffusion Modelscited by 15
  15. 2024On the Asymptotic Mean Square Error Optimality of Diffusion Modelscited by 10
  16. 2024Explicit Flow Matching: On The Theory of Flow Matching Algorithms with Applicationscited by 6
  17. 2025Optimizing Noise Schedules of Generative Models in High Dimensionsscited by 6
  18. 2025Implicit Regularisation in Diffusion Models: An Algorithm-Dependent Generalisation Analysiscited by 6
  19. 2024Nonequilbrium physics of generative diffusion modelscited by 5
  20. 2024On Mesa-Optimization in Autoregressively Trained Transformers: Emergence and Capabilitycited by 5
  21. 2025Localized Diffusion Modelscited by 5
  22. 2026A theory of learning data statistics in diffusion models, from easy to hardcited by 5
  23. 2026Weak Diffusion Priors Can Still Achieve Strong Inverse-Problem Performancecited by 4
  24. 2024Cryptographic Hardness of Score Estimationcited by 3
  25. 2025A Non-Asymptotic Convergent Analysis for Scored-Based Graph Generative Model via a System of Stochastic Differential Equationscited by 3
  26. 2026Emergence of Distortions in High-Dimensional Guided Diffusion Modelscited by 3
  27. 2026Asymptotic Learning Curves for Diffusion Models with Random Features Score and Manifold Datacited by 3
  28. 2024A solvable generative model with a linear, one-step denoisercited by 2
  29. 2025Computational bottlenecks for denoising diffusionscited by 2
  30. 2025Convergence Dynamics of Over-Parameterized Score Matching for a Single Gaussiancited by 2
  31. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributionscited by 2
  32. 2026Diffusion models recover accurate mixture weights despite score function insensitivitycited by 2
  33. 2026Denoising growth complexity: Data geometry and certified schedules for diffusion samplingcited by 2
  34. 2024Unrolled denoising networks provably learn optimal Bayesian inferencecited by 1
  35. 2025On the Collapse Errors Induced by the Deterministic Sampler for Diffusion Modelscited by 1
  36. 2026Multi-Subspace Multi-Modal Modeling for Diffusion Models: Estimation, Convergence and Mixture of Expertscited by 1
  37. 2024Generating Multi-Center Classifier via Conditional Gaussian Distributioncited by 0
  38. 2024Exploring Distortion Prior with Latent Diffusion Models for Remote Sensing Image Compressioncited by 0

50 most cited of 56 All 56 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Learning general Gaussian mixtures with efficient score matching
  2. 2025Optimal Convergence Analysis of DDPM for General Distributions
  3. 2026Global Convergence of Gradient Descent for Score Matching in Gaussian Mixtures via Reverse Fisher Divergence
  4. 2024The Unreasonable Effectiveness of Gaussian Score Approximation for Diffusion Models and its Applications
  5. 2024Diffusion Model Conditioning on Gaussian Mixture Model and Negative Gaussian Mixture Gradient
  6. 2025End-To-End Learning of Gaussian Mixture Priors for Diffusion Sampler
  7. 2025DDPM Score Matching and Distribution Learning
  8. 2026Transformers Learn the Optimal DDPM Denoiser for Multi-Token GMMs