# Hierarchical Maximum Entropy via the Renormalization Group

arXiv: https://arxiv.org/abs/2509.01424 · HTML: https://mlsanity.com/paper/2509.01424/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 34 references that are in the map.
- First posted on arXiv 1 Sep 2025.

## Builds on (most cited first, 4 of 4)

- [Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling](https://mlsanity.com/paper/2106.01357/) (2021)
- [User-friendly introduction to PAC-Bayes bounds](https://mlsanity.com/paper/2110.11216/) (2021)
- [Entropic Optimal Transport: Convergence of Potentials](https://mlsanity.com/paper/2104.11720/) (2021)
- [Demystifying ResNet](https://mlsanity.com/paper/1611.01186/) (2016)

## Built on by (most cited first, 1 of 1)

- [Equivalence Between Nested Gibbs Measures and Log-Linear Combinations of Gibbs Measures](https://mlsanity.com/paper/2609.19988/) (2026)

## Abstract (arXiv)

Hierarchical structures, which include multiple levels, are prevalent in statistical and machine-learning models as well as physical systems. Extending the foundational result that the maximum entropy distribution under mean constraints is given by the exponential Gibbs-Boltzmann form, we introduce the framework of "hierarchical maximum entropy" to address these multilevel models. We demonstrate that Pareto optimal distributions, which maximize entropies across all levels of hierarchical transformations, can be obtained via renormalization-group procedures from theoretical physics. This is achieved by formulating multilevel extensions of the Gibbs variational principle and the Donsker-Varadhan variational representation of entropy. Moreover, we explore settings with hierarchical invariances that significantly simplify the renormalization-group procedures, enhancing computational efficiency: quadratic modular loss functions, logarithmic loss functions, and nearest-neighbor loss functions. This is accomplished through the introduction of the concept of parameter flows, which serves as an analog to renormalization flows in renormalization group theory. This work connects ideas from probability theory, information theory, and statistical mechanics.