# Divergences induced by dual subtractive and divisive normalizations of exponential families and their convex deformations

arXiv: https://arxiv.org/abs/2312.12849 · HTML: https://mlsanity.com/paper/2312.12849/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 62 references that are in the map.
- First posted on arXiv 20 Dec 2023.

## Builds on (most cited first, 6 of 6)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [An elementary introduction to information geometry](https://mlsanity.com/paper/1808.08271/) (2018)
- [Relative Loss Bounds for On-line Density Estimation with the Exponential Family of Distributions](https://mlsanity.com/paper/1301.6677/) (2013)
- [Exponential Family Estimation via Adversarial Dynamics Embedding](https://mlsanity.com/paper/1904.12083/) (2019)
- [Fast approximations of the Jeffreys divergence between univariate Gaussian mixture models via exponential polynomial densities](https://mlsanity.com/paper/2107.05901/) (2021)
- [Variational Representations of Annealing Paths: Bregman Information under Monotonic Embedding](https://mlsanity.com/paper/2209.07481/) (2022)

## Built on by (most cited first, 2 of 2)

- [pyBregMan: A Python library for Bregman Manifolds](https://mlsanity.com/paper/2408.04175/) (2024)
- [Fast proxy centers for Jeffreys centroids: The Jeffreys-Fisher-Rao and the inductive Gauss-Bregman centers](https://mlsanity.com/paper/2410.14326/) (2024)

## Abstract (arXiv)

Exponential families are statistical models which are the workhorses in statistics, information theory, and machine learning among others. An exponential family can either be normalized subtractively by its cumulant or free energy function or equivalently normalized divisively by its partition function. Both subtractive and divisive normalizers are strictly convex and smooth functions inducing pairs of Bregman and Jensen divergences. It is well-known that skewed Bhattacharryya distances between probability densities of an exponential family amounts to skewed Jensen divergences induced by the cumulant function between their corresponding natural parameters, and in limit cases that the sided Kullback-Leibler divergences amount to reverse-sided Bregman divergences. In this paper, we first show that the α-divergences between unnormalized densities of an exponential family amounts to scaled α-skewed Jensen divergences induced by the partition function. We then show how comparative convexity with respect to a pair of quasi-arithmetic means allows to deform both convex functions and their arguments, and thereby define dually flat spaces with corresponding divergences when ordinary convexity is preserved.