mlsanityData to 7 Oct 2026

Do Heavy Tails Help Diffusion? On the Subtle Trade-off Between Initialization and Training

arXiv 2605.13175 ↗First posted 13 May 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent works have proposed incorporating heavy-tailed (HT) noise into diffusion- and flow-based generative models, with the goals of better recovering the tails of target distributions and improving generative diversity. This motivation is intuitive: if the data are heavy-tailed, HT noise may appear better matched than light-tailed (LT) Gaussian noise. However, replacing Gaussian noise by HT noise also changes the underlying estimation problem. In this paper, we revisit this paradigm through a combined theoretical and empirical study, establishing sampling-error bounds for two representative diffusion models driven by HT and LT noise. We show that HT noise makes the statistical estimation problem harder, leading to less favorable sampling-error bounds. We support these findings with experiments on synthetic and real-world datasets, empirically recovering the predicted error trade-off. Our results call into question a growing design trend in generative modeling and challenge the use of HT noise to improve rare-region exploration.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 30 of its 51 references that are in the map. First posted on arXiv 13 May 2026.

Builds on

30 of 51 references.

References · 30

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  8. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  9. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  10. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modelingcited by 594
  11. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptionscited by 440
  12. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
18 more
  1. 2022Convergence for score-based generative modeling with polynomial complexitycited by 176
  2. 2023KL Convergence Guarantees for Score diffusion models under minimal data assumptionscited by 61
  3. 2023Wasserstein Convergence Guarantees for a General Class of Score-Based Generative Modelscited by 55
  4. 2022Probability flow solution of the Fokker-Planck equationcited by 46
  5. 2023FinDiff: Diffusion Models for Financial Tabular Data Generationcited by 41
  6. 2024Heavy-Tailed Diffusion Modelscited by 34
  7. 2025Beyond Log-Concavity and Score Regularity: Improved Convergence Bounds for Score-Based Generative Models in W2-distancecited by 27
  8. 2024An analysis of the noise schedule for score-based generative modelscited by 25
  9. 2024Heavy-Tailed Diffusion with Denoising L'evy Probabilistic Modelscited by 16
  10. 2024Theoretical guarantees in KL for Diffusion Flow Matchingcited by 14
  11. 2025Mirror Flow Matching with Heavy-Tailed Priors for Generative Modeling on Convex Domainscited by 8
  12. 2024Tail calibration of probabilistic forecastscited by 7
  13. 2026Lipschitz regularity in Flow Matching and Diffusion Models: sharp sampling rates and functional inequalitiescited by 6
  14. 2025Wasserstein Convergence of Critically Damped Langevin Diffusionscited by 5
  15. 2025Generative modelling with jump-diffusionscited by 4
  16. 2026Diffusion Models with Heavy-Tailed Targets: Score Estimation and Sampling Guaranteescited by 4
  17. 2025Generating ensembles of spatially-coherent in-situ forecasts using flow matchingcited by 3
  18. 2025Diffusion Generative Models Meet Compressed Sensing, with Applications to Imaging and Financecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Self-Regulating Annealing in Heavy-Tailed Diffusion Models
  2. 2026Generalizing Score-based generative models for Heavy-tailed Distributions
  3. 2025Beyond Linear Diffusions: Improved Representations for Rare Conditional Generative Modeling
  4. 2024Not All Noises Are Created Equally:Diffusion Noise Selection and Optimization
  5. 2025Diffusion models under low-noise regime
  6. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models
  7. 2026Elucidating the SNR-t Bias of Diffusion Probabilistic Models
  8. 2026A Gaussian Perspective for Distributional Discrepancy in Generative Diffusion Models