# Equivalence Between Nested Gibbs Measures and Log-Linear Combinations of Gibbs Measures

arXiv: https://arxiv.org/abs/2609.19988 · HTML: https://mlsanity.com/paper/2609.19988/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 17 Sep 2026.

## Builds on (most cited first, 14 of 14)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Information-theoretic analysis of generalization capability of learning algorithms](https://mlsanity.com/paper/1705.07809/) (2017)
- [Social Learning and Distributed Hypothesis Testing](https://mlsanity.com/paper/1410.4307/) (2014)
- [What is the long-run distribution of stochastic gradient descent? A large deviations analysis](https://mlsanity.com/paper/2406.09241/) (2024)
- [On the Robustness to Misspecification of α-Posteriors and Their Variational Approximations](https://mlsanity.com/paper/2104.08324/) (2021)
- [Empirical Risk Minimization with Relative Entropy Regularization](https://mlsanity.com/paper/2211.06617/) (2022)
- [On the Validation of Gibbs Algorithms: Training Datasets, Test Datasets and their Aggregation](https://mlsanity.com/paper/2306.12380/) (2023)
- [An Information-Theoretic Approach to Generalization Theory](https://mlsanity.com/paper/2408.13275/) (2024)
- [Generalization Analysis of Machine Learning Algorithms via the Worst-Case Data-Generating Probability Measure](https://mlsanity.com/paper/2312.12236/) (2023)
- [The Method of Gaps: Exact Expressions for the Generalization Error of Supervised Learning Algorithms](https://mlsanity.com/paper/2411.12030/) (2024)
- [Variations on the Expectation due to Changes in the Probability Measure](https://mlsanity.com/paper/2502.02887/) (2025)
- [On the Generalization Error of Meta Learning for the Gibbs Algorithm](https://mlsanity.com/paper/2304.14332/) (2023)
- [Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms](https://mlsanity.com/paper/2604.20492/) (2026)
- [Hierarchical Maximum Entropy via the Renormalization Group](https://mlsanity.com/paper/2509.01424/) (2025)

## Abstract (arXiv)

In this paper, three operations on Gibbs probability measures are studied. The first operation, often referred to as renormalization, takes one Gibbs probability measure and generates a new Gibbs measure by normalizing a power of its density. This normalization has a twofold effect: it changes the regularization factor and concentrates the support within a subset of the original support. Interestingly, these effects can be independently controlled by different parameters. The second operation consists of a normalized log-linear combination of the densities of Gibbs probability measures. The third operation takes two Gibbs probability measures and changes the reference measure of the latter with the former. Hence, the former is said to be "nested" within the latter, yielding a new Gibbs probability measure. The resulting measures from the second and third operations are also Gibbs probability measures and are shown, respectively, to solve optimization problems involving the expectations of linear combinations of the objective functions of the given measures, subject to a relative entropy regularization. These optimization problems differ exclusively in the coefficients of the linear combinations. This leads to the conclusion that there exists a set of parameters for which nesting one Gibbs probability measure into another has the same effect as log-linearly combining them. These operations have relevant applications in statistical learning. As an example, a one-shot federated learning system in which clients send their locally trained Gibbs algorithms to the server for log-linear combination is shown to achieve the same performance as a Gibbs algorithm trained upon the aggregation of all local training datasets.