# On the Validation of Gibbs Algorithms: Training Datasets, Test Datasets and their Aggregation

arXiv: https://arxiv.org/abs/2306.12380 · HTML: https://mlsanity.com/paper/2306.12380/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 21 Jun 2023.

## Builds on (most cited first, 16 of 16)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Information-theoretic analysis of generalization capability of learning algorithms](https://mlsanity.com/paper/1705.07809/) (2017)
- [Statistical physics of inference: Thresholds and algorithms](https://mlsanity.com/paper/1511.02476/) (2015)
- [On the properties of variational approximations of Gibbs posteriors](https://mlsanity.com/paper/1506.04091/) (2015)
- [How much does your data exploration overfit? Controlling bias via information usage](https://mlsanity.com/paper/1511.05219/) (2015)
- [Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithms](https://mlsanity.com/paper/2004.12983/) (2020)
- [Chaining Mutual Information and Tightening Generalization Bounds](https://mlsanity.com/paper/1806.03803/) (2018)
- [Generalization Error Bounds Via R'enyi-, f-Divergences and Maximal Leakage](https://mlsanity.com/paper/1912.01439/) (2019)
- [PAC-Bayes unleashed: generalisation bounds with unbounded losses](https://mlsanity.com/paper/2006.07279/) (2020)
- [Tighter expected generalization error bounds via Wasserstein distance](https://mlsanity.com/paper/2101.09315/) (2021)
- [Jensen-Shannon Information Based Characterization of the Generalization Error of Learning Algorithms](https://mlsanity.com/paper/2010.12664/) (2020)
- [Still no free lunches: the price to pay for tighter PAC-Bayes bounds](https://mlsanity.com/paper/1910.04460/) (2019)
- [Tighter Expected Generalization Error Bounds via Convexity of Information Measures](https://mlsanity.com/paper/2202.12150/) (2022)
- [Empirical Risk Minimization with Relative Entropy Regularization: Optimality and Sensitivity Analysis](https://mlsanity.com/paper/2202.04385/) (2022)
- [Chaining Meets Chain Rule: Multilevel Entropic Regularization and Training of Neural Nets](https://mlsanity.com/paper/1906.11148/) (2019)
- [Analysis of the Relative Entropy Asymmetry in the Regularization of Empirical Risk Minimization](https://mlsanity.com/paper/2306.07123/) (2023)

## Built on by (most cited first, 10 of 10)

- [Generalization Analysis of Machine Learning Algorithms via the Worst-Case Data-Generating Probability Measure](https://mlsanity.com/paper/2312.12236/) (2023)
- [Equivalence of the Empirical Risk Minimization to Regularization on the Family of f-Divergences](https://mlsanity.com/paper/2402.00501/) (2024)
- [Asymmetry of the Relative Entropy in the Regularization of Empirical Risk Minimization](https://mlsanity.com/paper/2410.02833/) (2024)
- [The Method of Gaps: Exact Expressions for the Generalization Error of Supervised Learning Algorithms](https://mlsanity.com/paper/2411.12030/) (2024)
- [Variations on the Expectation due to Changes in the Probability Measure](https://mlsanity.com/paper/2502.02887/) (2025)
- [Decentralized Machine Learning with Centralized Performance Guarantees via Gibbs Algorithms](https://mlsanity.com/paper/2604.20492/) (2026)
- [A Dual Optimization View to Empirical Risk Minimization with f-Divergence Regularization](https://mlsanity.com/paper/2508.03314/) (2025)
- [Empirical Risk Minimization with f-Divergence Regularization](https://mlsanity.com/paper/2601.13191/) (2026)
- [Equivalence Between Nested Gibbs Measures and Log-Linear Combinations of Gibbs Measures](https://mlsanity.com/paper/2609.19988/) (2026)
- [Machine Unlearning for Gibbs Supervised Learning Algorithms](https://mlsanity.com/paper/2609.29409/) (2026)

## Abstract (arXiv)

The dependence on training data of the Gibbs algorithm (GA) is analytically characterized. By adopting the expected empirical risk as the performance metric, the sensitivity of the GA is obtained in closed form. In this case, sensitivity is the performance difference with respect to an arbitrary alternative algorithm. This description enables the development of explicit expressions involving the training errors and test errors of GAs trained with different datasets. Using these tools, dataset aggregation is studied and different figures of merit to evaluate the generalization capabilities of GAs are introduced. For particular sizes of such datasets and parameters of the GAs, a connection between Jeffrey's divergence, training and test errors is established.