mlsanityData to 7 Oct 2026

Concentration Inequalities for Stochastic Optimization of Unbounded Objective Functions with Application to Denoising Score Matching

arXiv 2502.08628 ↗First posted 12 Feb 2025stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We derive novel concentration inequalities that bound the statistical error for a large class of stochastic optimization problems, focusing on the case of unbounded objective functions. Our derivations utilize the following key tools: 1) A new form of McDiarmid's inequality that is based on sample-dependent one-component mean-difference bounds and which leads to a novel uniform law of large numbers result for unbounded functions. 2) A new Rademacher complexity bound for families of functions that satisfy an appropriate sample-dependent Lipschitz property, which allows for application to a large class of distributions with unbounded support. As an application of these results, we derive statistical error bounds for denoising score matching (DSM), an application that inherently requires one to consider unbounded objective functions and distributions with unbounded support, even in cases where the data distribution has bounded support. In addition, our results quantify the benefit of sample-reuse in algorithms that employ easily-sampled auxiliary random variables in addition to the training data, e.g., as in DSM, which uses auxiliary Gaussian random variables.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 34 references that are in the map. First posted on arXiv 12 Feb 2025.

Builds on

21 of 34 references.

References · 21

  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  2. 2017Improved Training of Wasserstein GANscited by 2,223
  3. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  4. 2016f-GAN: Training Generative Neural Samplers using Variational Divergence Minimizationcited by 504
  5. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localizationcited by 230
  6. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptionscited by 215
  7. 2017Generalization and Equilibrium in Generative Adversarial Nets (GANs)cited by 165
  8. 2023Diffusion Models are Minimax Optimal Distribution Estimatorscited by 159
  9. 2018How Well Generative Adversarial Networks Learn Distributionscited by 78
  10. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Modelscited by 64
  11. 2024Linear Convergence of Diffusion Models Under the Manifold Hypothesiscited by 42
  12. 2021An error analysis of generative adversarial networks for learning distributionscited by 35
9 more
  1. 2020Some Theoretical Insights into Wasserstein GANscited by 28
  2. 2019Bridging the Gap Between f-GANs and Wasserstein GANscited by 23
  3. 2023Improved Convergence of Score-Based Diffusion Models via Prediction-Correctioncited by 23
  4. 2013Concentration in unbounded metric spaces and algorithmic stabilitycited by 21
  5. 2024Score-based generative models are provably robust: an uncertainty quantification perspectivecited by 14
  6. 2020Cumulant GANcited by 9
  7. 2022Function-space regularized R'enyi divergencescited by 7
  8. 2024Robustness and Structure Preservation in Flow-Based Generative Models via Wasserstein Path-Space Divergencescited by 3
  9. 2024Statistical Error Bounds for GANs with Nonlinear Objective Functionalscited by 2

Built on by

Not cited yet.