# (f,Gamma)-Divergences: Interpolating between f-Divergences and Integral Probability Metrics

arXiv: https://arxiv.org/abs/2011.05953 · HTML: https://mlsanity.com/paper/2011.05953/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2020.

## Built on by (most cited first, 10 of 10)

- [On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributions](https://mlsanity.com/paper/2102.05485/) (2021)
- [Bayesian Posterior Perturbation Analysis with Integral Probability Metrics](https://mlsanity.com/paper/2303.01512/) (2023)
- [A Variational Framework for Residual-Based Adaptivity in Neural PDE Solvers and Operator Learning](https://mlsanity.com/paper/2509.14198/) (2025)
- [Moreau-Yosida f-divergences](https://mlsanity.com/paper/2102.13416/) (2021)
- [Improved Stochastic Optimization of LogSumExp](https://mlsanity.com/paper/2509.24894/) (2025)
- [Statistical and Geometrical properties of regularized Kernel Kullback-Leibler divergence](https://mlsanity.com/paper/2408.16543/) (2024)
- [Towards Understanding Gradient Dynamics of the Sliced-Wasserstein Distance via Critical Point Analysis](https://mlsanity.com/paper/2502.06525/) (2025)
- [Neural Measures for learning distributions of Random PDEs](https://mlsanity.com/paper/2507.01687/) (2025)
- [Fine-Tuning Generative Models for Extreme Events via CVaR-Penalized Wasserstein Gradient Flows](https://mlsanity.com/paper/2608.11544/) (2026)
- [Kernel Trace Distance: Quantum Statistical Metric between Measures through RKHS Density Operators](https://mlsanity.com/paper/2507.06055/) (2025)

## Abstract (arXiv)

We develop a rigorous and general framework for constructing information-theoretic divergences that subsume both f-divergences and integral probability metrics (IPMs), such as the 1-Wasserstein distance. We prove under which assumptions these divergences, hereafter referred to as (f,Gamma)-divergences, provide a notion of `distance' between probability measures and show that they can be expressed as a two-stage mass-redistribution/mass-transport process. The (f,Gamma)-divergences inherit features from IPMs, such as the ability to compare distributions which are not absolutely continuous, as well as from f-divergences, namely the strict concavity of their variational representations and the ability to control heavy-tailed distributions for particular choices of f. When combined, these features establish a divergence with improved properties for estimation, statistical learning, and uncertainty quantification applications. Using statistical learning as an example, we demonstrate their advantage in training generative adversarial networks (GANs) for heavy-tailed, not-absolutely continuous sample distributions. We also show improved performance and stability over gradient-penalized Wasserstein GAN in image generation.