# Information Processing Equalities and the Information-Risk Bridge

arXiv: https://arxiv.org/abs/2207.11987 · HTML: https://mlsanity.com/paper/2207.11987/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 148 references that are in the map.
- First posted on arXiv 25 Jul 2022.

## Builds on (most cited first, 18 of 18)

- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [A Theory of Usable Information Under Computational Constraints](https://mlsanity.com/paper/2002.10689/) (2020)
- [Learning with Symmetric Label Noise: The Importance of Being Unhinged](https://mlsanity.com/paper/1505.07634/) (2015)
- [Convex Geometry and Duality of Over-parameterized Neural Networks](https://mlsanity.com/paper/2002.11219/) (2020)
- [On the Discrimination-Generalization Tradeoff in GANs](https://mlsanity.com/paper/1711.02771/) (2017)
- [Fast rates in statistical and online learning](https://mlsanity.com/paper/1507.02592/) (2015)
- [Optimal Bounds between f-Divergences and Integral Probability Metrics](https://mlsanity.com/paper/2006.05973/) (2020)
- [Neural Estimation of Statistical Divergences](https://mlsanity.com/paper/2110.03652/) (2021)
- [Bridging the Gap Between f-GANs and Wasserstein GANs](https://mlsanity.com/paper/1910.09779/) (2019)
- [Martingale Methods for Sequential Estimation of Convex Functionals and Divergences](https://mlsanity.com/paper/2103.09267/) (2021)
- [Distributional Robustness with IPMs and links to Regularization and GANs](https://mlsanity.com/paper/2006.04349/) (2020)
- [Optimizing Variational Representations of Divergences and Accelerating their Statistical Estimation](https://mlsanity.com/paper/2006.08781/) (2020)
- [Smooth p-Wasserstein Distance: Structure, Empirical Approximation, and Statistical Applications](https://mlsanity.com/paper/2101.04039/) (2021)
- [Statistical and Topological Properties of Gaussian Smoothed Sliced Probability Divergences](https://mlsanity.com/paper/2110.10524/) (2021)
- [Moreau-Yosida f-divergences](https://mlsanity.com/paper/2102.13416/) (2021)

## Built on by (most cited first, 3 of 3)

- [Scoring Rules and Calibration for Imprecise Probabilities](https://mlsanity.com/paper/2410.23001/) (2024)
- [The Geometry and Calculus of Losses](https://mlsanity.com/paper/2209.00238/) (2022)
- [Comparing Corrupted Constrained Learning Problems](https://mlsanity.com/paper/2608.25745/) (2026)

## Abstract (arXiv)

We introduce two new classes of measures of information for statistical experiments which generalise and subsume φ-divergences, integral probability metrics, mathfrakN-distances (MMD), and (f,Gamma) divergences between two or more distributions. This enables us to derive a simple geometrical relationship between measures of information and the Bayes risk of a statistical decision problem, thus extending the variational φ-divergence representation to multiple distributions in an entirely symmetric manner. The new families of divergence are closed under the action of Markov operators which yields an information processing equality which is a refinement and generalisation of the classical data processing inequality. This equality gives insight into the significance of the choice of the hypothesis class in classical risk minimization.