# Do Heavy Tails Help Diffusion? On the Subtle Trade-off Between Initialization and Training

arXiv: https://arxiv.org/abs/2605.13175 · HTML: https://mlsanity.com/paper/2605.13175/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 13 May 2026.

## Builds on (most cited first, 30 of 30)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling](https://mlsanity.com/paper/2106.01357/) (2021)
- [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Convergence for score-based generative modeling with polynomial complexity](https://mlsanity.com/paper/2206.06227/) (2022)
- [KL Convergence Guarantees for Score diffusion models under minimal data assumptions](https://mlsanity.com/paper/2308.12240/) (2023)
- [Wasserstein Convergence Guarantees for a General Class of Score-Based Generative Models](https://mlsanity.com/paper/2311.11003/) (2023)
- [Probability flow solution of the Fokker-Planck equation](https://mlsanity.com/paper/2206.04642/) (2022)
- [FinDiff: Diffusion Models for Financial Tabular Data Generation](https://mlsanity.com/paper/2309.01472/) (2023)
- [Heavy-Tailed Diffusion Models](https://mlsanity.com/paper/2410.14171/) (2024)
- [Beyond Log-Concavity and Score Regularity: Improved Convergence Bounds for Score-Based Generative Models in W2-distance](https://mlsanity.com/paper/2501.02298/) (2025)
- [An analysis of the noise schedule for score-based generative models](https://mlsanity.com/paper/2402.04650/) (2024)
- [Heavy-Tailed Diffusion with Denoising L'evy Probabilistic Models](https://mlsanity.com/paper/2407.18609/) (2024)
- [Theoretical guarantees in KL for Diffusion Flow Matching](https://mlsanity.com/paper/2409.08311/) (2024)
- [Mirror Flow Matching with Heavy-Tailed Priors for Generative Modeling on Convex Domains](https://mlsanity.com/paper/2510.08929/) (2025)
- [Lipschitz regularity in Flow Matching and Diffusion Models: sharp sampling rates and functional inequalities](https://mlsanity.com/paper/2604.06065/) (2026)
- [Tail calibration of probabilistic forecasts](https://mlsanity.com/paper/2407.03167/) (2024)
- [Wasserstein Convergence of Critically Damped Langevin Diffusions](https://mlsanity.com/paper/2511.02419/) (2025)
- [Generative modelling with jump-diffusions](https://mlsanity.com/paper/2503.06558/) (2025)
- [Diffusion Models with Heavy-Tailed Targets: Score Estimation and Sampling Guarantees](https://mlsanity.com/paper/2601.06715/) (2026)
- [Generating ensembles of spatially-coherent in-situ forecasts using flow matching](https://mlsanity.com/paper/2504.03463/) (2025)
- [Diffusion Generative Models Meet Compressed Sensing, with Applications to Imaging and Finance](https://mlsanity.com/paper/2509.03898/) (2025)

## Built on by (most cited first, 3 of 3)

- [Radial Tail Repair: Post-Training Correction for Diffusion and Flow Models](https://mlsanity.com/paper/2610.16851/) (2026)
- [Annealed L'evy Diffusion via Gaussian Surrogate Reverse Dynamics](https://mlsanity.com/paper/2610.19692/) (2026)
- [From Global to Tail Convergence: Dynamics of α-Stable Score-Based Generative Models](https://mlsanity.com/paper/2610.19773/) (2026)

## Abstract (arXiv)

Recent works have proposed incorporating heavy-tailed (HT) noise into diffusion- and flow-based generative models, with the goals of better recovering the tails of target distributions and improving generative diversity. This motivation is intuitive: if the data are heavy-tailed, HT noise may appear better matched than light-tailed (LT) Gaussian noise. However, replacing Gaussian noise by HT noise also changes the underlying estimation problem. In this paper, we revisit this paradigm through a combined theoretical and empirical study, establishing sampling-error bounds for two representative diffusion models driven by HT and LT noise. We show that HT noise makes the statistical estimation problem harder, leading to less favorable sampling-error bounds. We support these findings with experiments on synthetic and real-world datasets, empirically recovering the predicted error trade-off. Our results call into question a growing design trend in generative modeling and challenge the use of HT noise to improve rare-region exploration.