# Heavy-Tailed Diffusion Models

arXiv: https://arxiv.org/abs/2410.14171 · HTML: https://mlsanity.com/paper/2410.14171/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 27 of its 61 references that are in the map.
- First posted on arXiv 18 Oct 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 27 of 27)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [Score-Based Generative Modeling with Critically-Damped Langevin Diffusion](https://mlsanity.com/paper/2112.07068/) (2021)
- [Generating Images with Perceptual Similarity Metrics based on Deep Networks](https://mlsanity.com/paper/1602.02644/) (2016)
- [GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium](https://mlsanity.com/paper/1706.08500/) (2017)
- [Residual Corrective Diffusion Modeling for Km-scale Atmospheric Downscaling](https://mlsanity.com/paper/2309.15214/) (2023)
- [DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents](https://mlsanity.com/paper/2201.00308/) (2022)
- [Kilometer-Scale Convection Allowing Model Emulation using Generative Diffusion Modeling](https://mlsanity.com/paper/2408.10958/) (2024)
- [Where to Diffuse, How to Diffuse, and How to Get Back: Automated Learning for Multivariate Diffusions](https://mlsanity.com/paper/2302.07261/) (2023)
- [Precipitation Downscaling with Spatiotemporal Video Diffusion](https://mlsanity.com/paper/2312.06071/) (2023)
- [t3-Variational Autoencoder: Learning Heavy-tailed Data with Student's t and Power Divergence](https://mlsanity.com/paper/2312.01133/) (2023)
- [Heavy-Tailed Diffusion with Denoising L'evy Probabilistic Models](https://mlsanity.com/paper/2407.18609/) (2024)
- [A Complete Recipe for Diffusion Generative Models](https://mlsanity.com/paper/2303.01748/) (2023)
- [Fast Samplers for Inverse Problems in Iterative Refinement Models](https://mlsanity.com/paper/2405.17673/) (2024)
- [Variational Inference based on Robust Divergences](https://mlsanity.com/paper/1710.06595/) (2017)
- [Marginal Tail-Adaptive Normalizing Flows](https://mlsanity.com/paper/2206.10311/) (2022)
- [Tails of Lipschitz Triangular Flows](https://mlsanity.com/paper/1907.04481/) (2019)

## Built on by (most cited first, 30 of 40)

- [Variational Control for Guidance in Diffusion Models](https://mlsanity.com/paper/2502.03686/) (2025)
- [Flow Density Control: Generative Optimization Beyond Entropy-Regularized Fine-Tuning](https://mlsanity.com/paper/2511.22640/) (2025)
- [Structured Diffusion Models with Mixture of Gaussians as Prior Distribution](https://mlsanity.com/paper/2410.19149/) (2024)
- [Mirror Flow Matching with Heavy-Tailed Priors for Generative Modeling on Convex Domains](https://mlsanity.com/paper/2510.08929/) (2025)
- [Random Walks with Tweedie: A Unified View of Score-Based Diffusion Models](https://mlsanity.com/paper/2411.18702/) (2024)
- [Go With the Flow: Fast Diffusion for Gaussian Mixture Models](https://mlsanity.com/paper/2412.09059/) (2024)
- [Generative modelling with jump-diffusions](https://mlsanity.com/paper/2503.06558/) (2025)
- [Controlled Latent Diffusion Models for 3D Porous Media Reconstruction](https://mlsanity.com/paper/2503.24083/) (2025)
- [Diffusion Models with Heavy-Tailed Targets: Score Estimation and Sampling Guarantees](https://mlsanity.com/paper/2601.06715/) (2026)
- [LightSBB-M: Bridging Schr"odinger and Bass for Generative Diffusion Modeling](https://mlsanity.com/paper/2601.19312/) (2026)
- [Calibrated Test-Time Guidance for Bayesian Inference](https://mlsanity.com/paper/2602.22428/) (2026)
- [Generating ensembles of spatially-coherent in-situ forecasts using flow matching](https://mlsanity.com/paper/2504.03463/) (2025)
- [The Recipe Matters More Than the Kitchen:Mathematical Foundations of the AI Weather Prediction Pipeline](https://mlsanity.com/paper/2604.01215/) (2026)
- [Generalized Wasserstein Flow Matching: Transport Plans, Everywhere, All at Once](https://mlsanity.com/paper/2605.08424/) (2026)
- [Do Heavy Tails Help Diffusion? On the Subtle Trade-off Between Initialization and Training](https://mlsanity.com/paper/2605.13175/) (2026)
- [Improved probabilistic regression using diffusion models](https://mlsanity.com/paper/2510.04583/) (2025)
- [E4GEN: Event-level Explainable Extreme-Enhanced Time-series Generation](https://mlsanity.com/paper/2606.01634/) (2026)
- [Sparse Data Diffusion for Scientific Simulations in Biology and Physics](https://mlsanity.com/paper/2502.02448/) (2025)
- [Offline Reinforcement Learning with Penalized Action Noise Injection](https://mlsanity.com/paper/2507.02356/) (2025)
- [Heteroscedasticity of Denoising Score Matching with Generalised Smooth Noise](https://mlsanity.com/paper/2508.01597/) (2025)
- [Sampling by averaging: A multiscale approach to score estimation](https://mlsanity.com/paper/2508.15069/) (2025)
- [Heavy-Tailed Class-Conditional Priors for Long-Tailed Generative Modeling](https://mlsanity.com/paper/2509.02154/) (2025)
- [Control-Augmented Autoregressive Diffusion for Data Assimilation](https://mlsanity.com/paper/2510.06637/) (2025)
- [Self-Regulating Annealing in Heavy-Tailed Diffusion Models](https://mlsanity.com/paper/2606.01645/) (2026)
- [Perspectives on Tsallis Statistics for Artificial Intelligence](https://mlsanity.com/paper/2608.01223/) (2026)
- [Fine-Tuning Generative Models for Extreme Events via CVaR-Penalized Wasserstein Gradient Flows](https://mlsanity.com/paper/2608.11544/) (2026)
- [Diffusion Models under Alternative Noise: Simplified Analysis and Sensitivity](https://mlsanity.com/paper/2506.08337/) (2025)
- [An effective potential for generative modelling with active matter](https://mlsanity.com/paper/2508.08146/) (2025)
- [Adapting Noise to Data: Generative Flows from 1D Processes](https://mlsanity.com/paper/2510.12636/) (2025)
- [Correcting Source Mismatch in Flow Matching with Radial-Angular Transport](https://mlsanity.com/paper/2604.04291/) (2026)

## Abstract (arXiv)

Diffusion models achieve state-of-the-art generation quality across many applications, but their ability to capture rare or extreme events in heavy-tailed distributions remains unclear. In this work, we show that traditional diffusion and flow-matching models with standard Gaussian priors fail to capture heavy-tailed behavior. We address this by repurposing the diffusion framework for heavy-tail estimation using multivariate Student-t distributions. We develop a tailored perturbation kernel and derive the denoising posterior based on the conditional Student-t distribution for the backward process. Inspired by γ-divergence for heavy-tailed distributions, we derive a training objective for heavy-tailed denoisers. The resulting framework introduces controllable tail generation using only a single scalar hyperparameter, making it easily tunable for diverse real-world distributions. As specific instantiations of our framework, we introduce t-EDM and t-Flow, extensions of existing diffusion and flow models that employ a Student-t prior. Remarkably, our approach is readily compatible with standard Gaussian diffusion models and requires only minimal code changes. Empirically, we show that our t-EDM and t-Flow outperform standard diffusion models in heavy-tail estimation on high-resolution weather datasets in which generating rare and extreme events is crucial.