# Diffusion Models with Heavy-Tailed Targets: Score Estimation and Sampling Guarantees

arXiv: https://arxiv.org/abs/2601.06715 · HTML: https://mlsanity.com/paper/2601.06715/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 29 of its 55 references that are in the map.
- First posted on arXiv 10 Jan 2026.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 29 of 29)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation](https://mlsanity.com/paper/2203.02923/) (2022)
- [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022)
- [Convergence of denoising diffusion models under the manifold hypothesis](https://mlsanity.com/paper/2208.05314/) (2022)
- [Generative Modeling with Denoising Auto-Encoders and Langevin Sampling](https://mlsanity.com/paper/2002.00107/) (2020)
- [Protein Structure and Sequence Generation with Equivariant Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2205.15019/) (2022)
- [Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions](https://mlsanity.com/paper/2409.18804/) (2024)
- [Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial](https://mlsanity.com/paper/2402.07487/) (2024)
- [Learning general Gaussian mixtures with efficient score matching](https://mlsanity.com/paper/2404.18893/) (2024)
- [Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization](https://mlsanity.com/paper/2401.15604/) (2024)
- [Learning Mixtures of Gaussians Using Diffusion Models](https://mlsanity.com/paper/2404.18869/) (2024)
- [Heavy-Tailed Diffusion Models](https://mlsanity.com/paper/2410.14171/) (2024)
- [Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Models](https://mlsanity.com/paper/2410.04760/) (2024)
- [Faster Diffusion Sampling with Randomized Midpoints: Sequential and Parallel](https://mlsanity.com/paper/2406.00924/) (2024)
- [From optimal score matching to optimal sampling](https://mlsanity.com/paper/2409.07032/) (2024)
- [Denoising Diffusion Gamma Models](https://mlsanity.com/paper/2110.05948/) (2021)
- [Minimax Optimality of the Probability Flow ODE for Diffusion Models](https://mlsanity.com/paper/2503.09583/) (2025)
- [Instance-dependent Convergence Theory for Diffusion Models](https://mlsanity.com/paper/2410.13738/) (2024)
- [Generalization error bound for denoising score matching under relaxed manifold assumption](https://mlsanity.com/paper/2502.13662/) (2025)
- [Heavy-tailed denoising score matching](https://mlsanity.com/paper/2112.09788/) (2021)
- [Advancing Wasserstein Convergence Analysis of Score-Based Models: Insights from Discretization and Second-Order Acceleration](https://mlsanity.com/paper/2502.04849/) (2025)
- [Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributions](https://mlsanity.com/paper/2402.08082/) (2024)
- [Non-asymptotic Analysis of Diffusion Annealed Langevin Monte Carlo for Generative Modelling](https://mlsanity.com/paper/2502.09306/) (2025)
- [Fast Convergence for High-Order ODE Solvers in Diffusion Probabilistic Models](https://mlsanity.com/paper/2506.13061/) (2025)
- [DDPM Score Matching and Distribution Learning](https://mlsanity.com/paper/2504.05161/) (2025)
- [Implicit score matching meets denoising score matching: improved rates of convergence and log-density Hessian estimation](https://mlsanity.com/paper/2512.24378/) (2025)

## Built on by (most cited first, 5 of 5)

- [Generalizing Score-based generative models for Heavy-tailed Distributions](https://mlsanity.com/paper/2603.00772/) (2026)
- [Do Heavy Tails Help Diffusion? On the Subtle Trade-off Between Initialization and Training](https://mlsanity.com/paper/2605.13175/) (2026)
- [Diachronic Sample Integration: Robust Tail-Risk Estimation with Generative Models](https://mlsanity.com/paper/2607.10810/) (2026)
- [Robustness of Diffusion Models under Distribution Shift](https://mlsanity.com/paper/2609.27546/) (2026)
- [Annealed L'evy Diffusion via Gaussian Surrogate Reverse Dynamics](https://mlsanity.com/paper/2610.19692/) (2026)

## Abstract (arXiv)

Score-based diffusion models have become a powerful framework for generative modeling, with score estimation as a central statistical bottleneck. Existing guarantees for score estimation largely focus on light-tailed targets or rely on restrictive assumptions such as compact support, which are often violated by heavy-tailed data in practice. In this work, we study conventional (Gaussian) score-based diffusion models when the target distribution is heavy-tailed and belongs to a Sobolev class with smoothness parameter β>0. We consider both exponential and polynomial tail decay, indexed by a tail parameter γ. Using kernel density estimation, we derive sharp minimax rates for score estimation, revealing a qualitative dichotomy: under exponential tails, the rate matches the light-tailed case up to polylogarithmic factors, whereas under polynomial tails the rate depends explicitly on γ. We further provide sampling guarantees for the associated continuous reverse dynamics. In total variation, the generated distribution converges at the minimax optimal rate n-β/(2β+d) under exponential tails (up to logarithmic factors), and at a γ-dependent rate under polynomial tails. Whether the latter sampling rate is minimax optimal remains an open question. These results characterize the statistical limits of score estimation and the resulting sampling accuracy for heavy-tailed targets, extending diffusion theory beyond the light-tailed setting.