# Heavy-Tailed Diffusion with Denoising L'evy Probabilistic Models

arXiv: https://arxiv.org/abs/2407.18609 · HTML: https://mlsanity.com/paper/2407.18609/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 26 Jul 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 18 of 18)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [Mean estimation and regression under heavy-tailed distributions--a survey](https://mlsanity.com/paper/1906.04280/) (2019)
- [CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Sampling](https://mlsanity.com/paper/2310.17347/) (2023)
- [Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise](https://mlsanity.com/paper/2002.05685/) (2020)
- [Denoising Diffusion Gamma Models](https://mlsanity.com/paper/2110.05948/) (2021)
- [Fractional Langevin Monte Carlo: Exploring L'evy Driven Stochastic Differential Equations for Markov Chain Monte Carlo](https://mlsanity.com/paper/1706.03649/) (2017)
- [Generative Fractional Diffusion Models](https://mlsanity.com/paper/2310.17638/) (2023)

## Built on by (most cited first, 18 of 18)

- [Heavy-Tailed Diffusion Models](https://mlsanity.com/paper/2410.14171/) (2024)
- [Generative modelling with jump-diffusions](https://mlsanity.com/paper/2503.06558/) (2025)
- [Do Heavy Tails Help Diffusion? On the Subtle Trade-off Between Initialization and Training](https://mlsanity.com/paper/2605.13175/) (2026)
- [DGenCTR: Towards a Universal Generative Paradigm for Click-Through Rate Prediction via Discrete Diffusion](https://mlsanity.com/paper/2508.14500/) (2025)
- [E4GEN: Event-level Explainable Extreme-Enhanced Time-series Generation](https://mlsanity.com/paper/2606.01634/) (2026)
- [Score-based Metropolis-Hastings for Fractional Langevin Algorithms](https://mlsanity.com/paper/2602.00835/) (2026)
- [Self-Improving VLA Policies: Selected Diffusion Noise for Spurious-Robust Action Smoothing](https://mlsanity.com/paper/2606.14084/) (2026)
- [Perspectives on Tsallis Statistics for Artificial Intelligence](https://mlsanity.com/paper/2608.01223/) (2026)
- [Generator-Guided Inverse Sampling for L'evy-Driven Generative Models](https://mlsanity.com/paper/2608.10384/) (2026)
- [Conditional Deep Levy Models for Exotic Derivatives: History-Aware Path Generation and P-Q Payoff Diagnostics](https://mlsanity.com/paper/2509.13374/) (2025)
- [Adapting Noise to Data: Generative Flows from 1D Processes](https://mlsanity.com/paper/2510.12636/) (2025)
- [HyperNet-Adaptation for Diffusion-Based Test Case Generation](https://mlsanity.com/paper/2601.15041/) (2026)
- [Sampling from multi-modal distributions on Riemannian manifolds with training-free stochastic interpolants](https://mlsanity.com/paper/2602.00641/) (2026)
- [Beyond Two-stage Diffusion TTS: Joint Structure and Content Refinement via Jump Diffusion](https://mlsanity.com/paper/2603.14032/) (2026)
- [Heavy-Tailed Flow Matching via Random Clocks](https://mlsanity.com/paper/2607.13841/) (2026)
- [LATS: Levy Adaptive Tree Sampling for Feedback-Driven Diverse Target Discovery](https://mlsanity.com/paper/2609.06761/) (2026)
- [Annealed L'evy Diffusion via Gaussian Surrogate Reverse Dynamics](https://mlsanity.com/paper/2610.19692/) (2026)
- [From Global to Tail Convergence: Dynamics of α-Stable Score-Based Generative Models](https://mlsanity.com/paper/2610.19773/) (2026)

## Abstract (arXiv)

Exploring noise distributions beyond Gaussian in diffusion models remains an open challenge. While Gaussian-based models succeed within a unified SDE framework, recent studies suggest that heavy-tailed noise distributions, like α-stable distributions, may better handle mode collapse and effectively manage datasets exhibiting class imbalance, heavy tails, or prominent outliers. Recently, Yoon et al. (NeurIPS 2023), presented the L'evy-Ito model (LIM), directly extending the SDE-based framework to a class of heavy-tailed SDEs, where the injected noise followed an α-stable distribution, a rich class of heavy-tailed distributions. However, the LIM framework relies on highly involved mathematical techniques with limited flexibility, potentially hindering broader adoption and further development. In this study, instead of starting from the SDE formulation, we extend the denoising diffusion probabilistic model (DDPM) by replacing the Gaussian noise with α-stable noise. By using only elementary proof techniques, the proposed approach, Denoising L'evy Probabilistic Models (DLPM), boils down to vanilla DDPM with minor modifications. As opposed to the Gaussian case, DLPM and LIM yield different training algorithms and different backward processes, leading to distinct sampling algorithms. These fundamental differences translate favorably for DLPM as compared to LIM: our experiments show improvements in coverage of data distribution tails, better robustness to unbalanced datasets, and improved computation times requiring smaller number of backward steps.