# Exponential families from a single KL identity

arXiv: https://arxiv.org/abs/2604.28036 · HTML: https://mlsanity.com/paper/2604.28036/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 22 references that are in the map.
- First posted on arXiv 30 Apr 2026.

## Builds on (most cited first, 1 of 1)

- [RL with KL penalties is better viewed as Bayesian inference](https://mlsanity.com/paper/2205.11275/) (2022)

## Built on by (most cited first, 1 of 1)

- [Binary Rewards and Reinforcement Learning: Fundamental Challenges](https://mlsanity.com/paper/2605.02375/) (2026)

## Abstract (arXiv)

Exponential families encompass the distributions central to modern machine learning -- softmax, Gaussians, and Boltzmann distributions -- and underlie the theory of variational inference, entropy-regularized reinforcement learning, and RLHF. We isolate a simple identity for exponential families that expresses the KL difference KL(q | pλ2) - KL(q | pλ1) in terms of the log-partition function A(λ) and the moment μq. Remarkably, this identity together with the single fact that KL ≥ 0 (with equality iff p = q) suffices, by direct substitution and rearrangement, to derive a cluster of results that are classically obtained by separate, heavier arguments: a generalized three-point identity for arbitrary reference distributions, Pythagorean theorems for I-projections and reverse I-projections, convexity of the log-partition function, identification of its Legendre dual in KL terms, the Gibbs variational principle, and the explicit optimizer in KL-regularized reward maximization, including the exponential tilting formula underlying entropy-regularized control and RLHF. Beyond these purely algebraic consequences, standard analytic arguments recover the gradient formula for the log-partition function, the Bregman representation of within-family KL divergence, and the surjectivity of the moment map. The note is self-contained.