mlsanityData to 7 Oct 2026

Jensen-Shannon Information Based Characterization of the Generalization Error of Learning Algorithms

arXiv 2010.12664 ↗First posted 23 Oct 2020cs.IT math.IT math.STcited by 19

Not on the map. Why?

Abstract (arXiv)

Generalization error bounds are critical to understanding the performance of machine learning models. In this work, we propose a new information-theoretic based generalization error upper bound applicable to supervised learning scenarios. We show that our general bound can specialize in various previous bounds. We also show that our general bound can be specialized under some conditions to a new bound involving the Jensen-Shannon information between a random variable modelling the set of training samples and another random variable modelling the hypothesis. We also prove that our bound can be tighter than mutual information-based bounds under some conditions.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 7; 2023: 3; 2024: 2; 2025: 0; 2026: 2.

19 papers.

Sort:

  1. 2022An Information-theoretical Approach to Semi-supervised Learning under Covariate-shiftcited by 18
  2. 2022Understanding Generalization via Leave-One-Out Conditional Mutual Informationcited by 16
  3. 2021Generalization Bounds for Noisy Iterative Algorithms Using Properties of Additive Noise Channelscited by 15
  4. 2021Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithmcited by 14
  5. 2022Empirical Risk Minimization with Relative Entropy Regularizationcited by 13
  6. 2021Information-Theoretic Bounds on the Moments of the Generalization Error of Learning Algorithmscited by 12
  7. 2022Tighter Expected Generalization Error Bounds via Convexity of Information Measurescited by 11
  8. 2021An Information-Theoretic Analysis of the Impact of Task Similarity on Meta-Learningcited by 10
  9. 2023On the Validation of Gibbs Algorithms: Training Datasets, Test Datasets and their Aggregationcited by 10
  10. 2022f-divergences and their applications in lossy compression and bounding generalization errorcited by 7
  11. 2023Generalization Analysis of Machine Learning Algorithms via the Worst-Case Data-Generating Probability Measurecited by 7
  12. 2022Learning Algorithm Generalization Error Bounds via Auxiliary Distributionscited by 6
7 more
  1. 2024The Method of Gaps: Exact Expressions for the Generalization Error of Supervised Learning Algorithmscited by 5
  2. 2024Generalization and Robustness of the Tilted Empirical Riskcited by 4
  3. 2021Characterizing the Generalization Error of Gibbs Algorithm with Symmetrized KL informationcited by 3
  4. 2022Formal limitations of sample-wise information-theoretic generalization boundscited by 3
  5. 2023Information-Theoretic Generalization Bounds for Transductive Learning and its Applicationscited by 2
  6. 2026Empirical Risk Minimization with f-Divergence Regularizationcited by 0
  7. 2026Not all Jensen-Shannon Divergence Estimators are Equalcited by 0