# Information from coincidences

arXiv: https://arxiv.org/abs/2606.25042 · HTML: https://mlsanity.com/paper/2606.25042/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 98 references that are in the map.
- First posted on arXiv 23 Jun 2026.

## Builds on (most cited first, 4 of 4)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [Sharp finite-sample concentration of independent variables](https://mlsanity.com/paper/2008.13293/) (2020)
- [Entropy, concentration, and learning: a statistical mechanics primer](https://mlsanity.com/paper/2409.18630/) (2024)

## Built on by (most cited first, 4 of 4)

- [Adaptive Bayes exactly tracks information over intrinsic time](https://mlsanity.com/paper/2607.08789/) (2026)
- [All you need is log](https://mlsanity.com/paper/2606.27349/) (2026)
- [The concentration game: Bayesian updating, regret, and information](https://mlsanity.com/paper/2608.18061/) (2026)
- [Information on trajectories: martingales and random times](https://mlsanity.com/paper/2608.20337/) (2026)

## Abstract (arXiv)

We prove a single algebraic mixed coincidence identity that unifies a broad swath of information-theoretic variational results. For any family of priors πi and real exponents  αi , the log of the mixed count Ex~nu!left[prodi=1W πiαi(x)right] is simultaneously a Boltzmann coincidence weight, an exponential-family normalizer, a maximum-entropy value, and a KL-barycenter optimum. The identity yields a unified derivation of classical cornerstones of information theory: concentration of empirical distributions (Sanov-type decompositions and Gibbs conditioning), hypothesis-testing error exponents (Chernoff information and its multi-way analogue), change-of-measure inequalities (Donsker-Varadhan and PAC-Bayes), and laws governing rare-pattern coincidences (Erdos-Renyi run-length, iterative guesswork, rate-distortion, and birthday thresholds). Each is recovered as a specialization of the same algebraic equality. It strictly generalizes the classical Renyi entropy and divergence variational formulas (one and two priors respectively) to a W-prior simplex, and holds for unnormalized and continuum-indexed priors. Among its consequences are an exact multi-prior PAC-Bayes penalty that subtracts an explicit "coincidence bonus" from the usual single-prior posterior penalty, and the asymptotic MAP error exponent for W-ary hypothesis testing as an edge-restricted simplex optimum. We demonstrate the calculus at scale on two large alphabets encoding richly modeled sequential languages: on language-model next-token predictives where we recover contrastive decoding, and on human genomic regulatory sequence where it separates correlated from diverse prior families along a sliding-window trace.