# Spread Divergence

arXiv: https://arxiv.org/abs/1811.08968 · HTML: https://mlsanity.com/paper/1811.08968/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Nov 2018.

## Built on by (most cited first, 23 of 23)

- [Information-Theoretic Generalization Bounds for Stochastic Gradient Descent](https://mlsanity.com/paper/2102.00931/) (2021)
- [Diagnosing and Fixing Manifold Overfitting in Deep Generative Models](https://mlsanity.com/paper/2204.07172/) (2022)
- [Free-form Flows: Make Any Architecture a Normalizing Flow](https://mlsanity.com/paper/2310.16624/) (2023)
- [Diffusive Gibbs Sampling](https://mlsanity.com/paper/2402.03008/) (2024)
- [Training Neural Samplers with Reverse Diffusive KL Divergence](https://mlsanity.com/paper/2410.12456/) (2024)
- [Limitations of Information-Theoretic Generalization Bounds for Gradient Descent Methods in Stochastic Convex Optimization](https://mlsanity.com/paper/2212.13556/) (2022)
- [Towards Healing the Blindness of Score Matching](https://mlsanity.com/paper/2209.07396/) (2022)
- [Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matching](https://mlsanity.com/paper/2406.10808/) (2024)
- [Generalization Gap in Amortized Inference](https://mlsanity.com/paper/2205.11640/) (2022)
- [The Score-Difference Flow for Implicit Generative Modeling](https://mlsanity.com/paper/2304.12906/) (2023)
- [Energy Discrepancies: A Score-Independent Loss for Energy-Based Models](https://mlsanity.com/paper/2307.06431/) (2023)
- [DiffRatio: Training One-Step Diffusion Models Without Teacher Supervision](https://mlsanity.com/paper/2502.08005/) (2025)
- [Moment Matching Denoising Gibbs Sampling](https://mlsanity.com/paper/2305.11650/) (2023)
- [Denoising Deep Generative Models](https://mlsanity.com/paper/2212.01265/) (2022)
- [Estimating the Rate-Distortion Function by Wasserstein Gradient Descent](https://mlsanity.com/paper/2310.18908/) (2023)
- [Out-of-Distribution Detection with Class Ratio Estimation](https://mlsanity.com/paper/2206.03955/) (2022)
- [Using Perturbation to Improve Goodness-of-Fit Tests based on Kernelized Stein Discrepancy](https://mlsanity.com/paper/2304.14762/) (2023)
- [Conditional COT-GAN for Video Prediction with Kernel Smoothing](https://mlsanity.com/paper/2106.05658/) (2021)
- [On Convolutions, Intrinsic Dimension, and Diffusion Models](https://mlsanity.com/paper/2506.20705/) (2025)
- [VarDiU: A Variational Diffusive Upper Bound for One-Step Diffusion Distillation](https://mlsanity.com/paper/2508.20646/) (2025)
- [Guidance for Prior Change via Density Ratio Estimation](https://mlsanity.com/paper/2608.21729/) (2026)
- [Variational Autoencoders: A Harmonic Perspective](https://mlsanity.com/paper/2105.14866/) (2021)
- [Integrated Weak Learning](https://mlsanity.com/paper/2206.09496/) (2022)

## Abstract (arXiv)

For distributions mathbbP and mathbbQ with different supports or undefined densities, the divergence textrmD(mathbbP||mathbbQ) may not exist. We define a Spread Divergence tildetextrmD(mathbbP||mathbbQ) on modified mathbbP and mathbbQ and describe sufficient conditions for the existence of such a divergence. We demonstrate how to maximize the discriminatory power of a given divergence by parameterizing and learning the spread. We also give examples of using a Spread Divergence to train implicit generative models, including linear models (Independent Components Analysis) and non-linear models (Deep Generative Networks).