mlsanityData to 14 Oct 2026

On Random Subset Generalization Error Bounds and the Stochastic Gradient Langevin Dynamics Algorithm

arXiv 2010.10994 ↗First posted 21 Oct 2020cs.IT math.IT stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

In this work, we unify several expected generalization error bounds based on random subsets using the framework developed by Hellstr"om and Durisi [1]. First, we recover the bounds based on the individual sample mutual information from Bu et al. [2] and on a random subset of the dataset from Negrea et al. [3]. Then, we introduce their new, analogous bounds in the randomized subsample setting from Steinke and Zakynthinou [4], and we identify some limitations of the framework. Finally, we extend the bounds from Haghifam et al. [5] for Langevin dynamics to stochastic gradient Langevin dynamics and we refine them for loss functions with potentially large gradient norms.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 6; 2022: 8; 2023: 7; 2024: 4; 2025: 2; 2026: 1.

28 papers.

Sort:

  1. 2021Information-Theoretic Generalization Bounds for Stochastic Gradient Descentcited by 84
  2. 2021Information-theoretic generalization bounds for black-box learning algorithmscited by 49
  3. 2021Tighter expected generalization error bounds via Wasserstein distancecited by 43
  4. 2021On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implicationscited by 31
  5. 2021Towards a Unified Information-Theoretic Framework for Generalizationcited by 30
  6. 2022A New Family of Generalization Bounds Using Samplewise Evaluated CMIcited by 28
  7. 2023Tighter Information-Theoretic Generalization Bounds from Supersamplescited by 22
  8. 2022Limitations of Information-Theoretic Generalization Bounds for Gradient Descent Methods in Stochastic Convex Optimizationcited by 19
  9. 2023Information Theoretic Lower Bounds for Information Theoretic Upper Boundscited by 17
  10. 2022Understanding Generalization via Leave-One-Out Conditional Mutual Informationcited by 16
  11. 2021Generalization Bounds for Noisy Iterative Algorithms Using Properties of Additive Noise Channelscited by 15
  12. 2022Information-Theoretic Analysis of Unsupervised Domain Adaptationcited by 15
16 more
  1. 2024Information Complexity of Stochastic Convex Optimization: Applications to Generalization and Memorizationcited by 14
  2. 2022Evaluated CMI Bounds for Meta Learning: Tightness and Expressivenesscited by 11
  3. 2024Minimum Description Length and Generalization Guarantees for Representation Learningcited by 11
  4. 2022Chained Generalisation Boundscited by 10
  5. 2022Stability Based Generalization Bounds for Exponential Family Langevin Dynamicscited by 8
  6. 2023Sample-Conditioned Hypothesis Stability Sharpens Information-Theoretic Generalization Boundscited by 6
  7. 2023Fast Rate Information-theoretic Bounds on Generalization Errorscited by 5
  8. 2023Comparing Comparators in Generalization Boundscited by 5
  9. 2023Exactly Tight Information-Theoretic Generalization Error Bound for the Quadratic Gaussian Problemcited by 4
  10. 2024Generalization Bounds via Conditional f-Informationcited by 4
  11. 2022Formal limitations of sample-wise information-theoretic generalization boundscited by 3
  12. 2024An Information-Theoretic Framework for Out-of-Distribution Generalization with Applications to Stochastic Gradient Langevin Dynamicscited by 3
  13. 2023Information-Theoretic Generalization Bounds for Transductive Learning and its Applicationscited by 2
  14. 2025Information-Theoretic Generalization Bounds of Replay-based Continual Learningcited by 2
  15. 2025Beyond Real Data: Synthetic Data through the Lens of Regularizationcited by 2
  16. 2026Block-Sample MAC-Bayes Generalization Boundscited by 0