# Optimal Bounds between f-Divergences and Integral Probability Metrics

arXiv: https://arxiv.org/abs/2006.05973 · HTML: https://mlsanity.com/paper/2006.05973/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2020.

## Built on by (most cited first, 27 of 27)

- [On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributions](https://mlsanity.com/paper/2102.05485/) (2021)
- [Understanding Contrastive Learning via Distributionally Robust Optimization](https://mlsanity.com/paper/2310.11048/) (2023)
- [Martingale Methods for Sequential Estimation of Convex Functionals and Divergences](https://mlsanity.com/paper/2103.09267/) (2021)
- [Information-Theoretic Analysis of Unsupervised Domain Adaptation](https://mlsanity.com/paper/2210.00706/) (2022)
- [Wasserstein Gradient Flows for Moreau Envelopes of f-Divergences in Reproducing Kernel Hilbert Spaces](https://mlsanity.com/paper/2402.04613/) (2024)
- [Distributional Reinforcement Learning with Regularized Wasserstein Loss](https://mlsanity.com/paper/2202.00769/) (2022)
- [Light Unbalanced Optimal Transport](https://mlsanity.com/paper/2303.07988/) (2023)
- [On f-Divergence Principled Domain Adaptation: An Improved Framework](https://mlsanity.com/paper/2402.01887/) (2024)
- [Moreau-Yosida f-divergences](https://mlsanity.com/paper/2102.13416/) (2021)
- [Intrinsic Benefits of Categorical Distributional Loss: Uncertainty-aware Regularized Exploration in Reinforcement Learning](https://mlsanity.com/paper/2110.03155/) (2021)
- [Outcome Assumptions and Duality Theory for Balancing Weights](https://mlsanity.com/paper/2203.09557/) (2022)
- [Optimal transport with f-divergence regularization and generalized Sinkhorn algorithm](https://mlsanity.com/paper/2105.14337/) (2021)
- [Generalization Bounds via Conditional f-Information](https://mlsanity.com/paper/2410.22887/) (2024)
- [Proofs for Folklore Theorems on the Radon-Nikodym Derivative](https://mlsanity.com/paper/2501.18374/) (2025)
- [Information Processing Equalities and the Information-Risk Bridge](https://mlsanity.com/paper/2207.11987/) (2022)
- [An Information-Theoretic Framework for Out-of-Distribution Generalization with Applications to Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2403.19895/) (2024)
- [Decision Making under the Exponential Family: Distributionally Robust Optimisation with Bayesian Ambiguity Sets](https://mlsanity.com/paper/2411.16829/) (2024)
- [Variations on the Expectation due to Changes in the Probability Measure](https://mlsanity.com/paper/2502.02887/) (2025)
- [Online non-parametric likelihood-ratio estimation by Pearson-divergence functional minimization](https://mlsanity.com/paper/2311.01900/) (2023)
- [Distributionally Robust Optimisation with Bayesian Ambiguity Sets](https://mlsanity.com/paper/2409.03492/) (2024)
- [BOLT: Universal Classifier Training via a Bayes-Error Upper Bound](https://mlsanity.com/paper/2501.07754/) (2025)
- [Addressing Label Shift in Distributed Learning via Entropy Regularization](https://mlsanity.com/paper/2502.02544/) (2025)
- [Unified Perspective on Probability Divergence via Maximum Likelihood Density Ratio Estimation: Bridging KL-Divergence and Integral Probability Metrics](https://mlsanity.com/paper/2201.13127/) (2022)
- [R-divergence for Estimating Model-oriented Distribution Discrepancy](https://mlsanity.com/paper/2310.01109/) (2023)
- [Seeking Flat Minima over Diverse Surrogates for Improved Adversarial Transferability: A Theoretical Framework and Algorithmic Instantiation](https://mlsanity.com/paper/2504.16474/) (2025)
- [Optimal Stability of KL Divergence under Gaussian Perturbations](https://mlsanity.com/paper/2604.11026/) (2026)
- [Copula Transformations for Data-Consistent Inversion](https://mlsanity.com/paper/2609.02832/) (2026)

## Abstract (arXiv)

The families of f-divergences (e.g. the Kullback-Leibler divergence) and Integral Probability Metrics (e.g. total variation distance or maximum mean discrepancies) are widely used to quantify the similarity between probability distributions. In this work, we systematically study the relationship between these two families from the perspective of convex duality. Starting from a tight variational representation of the f-divergence, we derive a generalization of the moment-generating function, which we show exactly characterizes the best lower bound of the f-divergence as a function of a given IPM. Using this characterization, we obtain new bounds while also recovering in a unified manner well-known results, such as Hoeffding's lemma, Pinsker's inequality and its extension to subgaussian functions, and the Hammersley-Chapman-Robbins bound. This characterization also allows us to prove new results on topological properties of the divergence which may be of independent interest.