mlsanityData to 14 Oct 2026

Integral Probability Metrics PAC-Bayes Bounds

arXiv 2207.00614 ↗First posted 1 Jul 2022stat.ML cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

We present a PAC-Bayes-style generalization bound which enables the replacement of the KL-divergence with a variety of Integral Probability Metrics (IPM). We provide instances of this bound with the IPM being the total variation metric and the Wasserstein distance. A notable feature of the obtained bounds is that they naturally interpolate between classical uniform convergence bounds in the worst case (when the prior and posterior are far away from each other), and improved bounds in favorable cases (when the posterior and prior are close). This illustrates the possibility of reinforcing classical generalization bounds with algorithm- and data-dependent components, thus making them more suitable to analyze algorithms that use a large hypothesis space.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 26 of its 75 references that are in the map. First posted on arXiv 1 Jul 2022.

Builds on

26 of 75 references.

References · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
  4. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  5. 2017Exploring Generalization in Deep Learningcited by 600
  6. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  7. 2019Fantastic Generalization Measures and Where to Find Themcited by 513
  8. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  9. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  10. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 224
  11. 2019A Primer on PAC-Bayesian Learningcited by 132
  12. 2017Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theorycited by 76
14 more
  1. 2019Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentationcited by 71
  2. 2020PAC-Bayes Analysis Beyond the Usual Boundscited by 63
  3. 2019Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resiliencecited by 51
  4. 2016Simpler PAC-Bayesian Bounds for Hostile Datacited by 46
  5. 2021Tighter expected generalization error bounds via Wasserstein distancecited by 43
  6. 2019Meta Dynamic Pricing: Transfer Learning Across Experimentscited by 42
  7. 2022Generalization Bounds via Convex Analysiscited by 27
  8. 2020A Limitation of the PAC-Bayes Frameworkcited by 25
  9. 2021Measuring Generalization with Optimal Transportcited by 25
  10. 2019Fast-rate PAC-Bayes Generalization Bounds via Shifted Rademacher Processescited by 20
  11. 2022Tighter Expected Generalization Error Bounds via Convexity of Information Measurescited by 11
  12. 2022Change of measure through the Legendre transformcited by 10
  13. 2021Uniform Convergence, Adversarial Spheres and a Simple Remedycited by 8
  14. 2019PAC-Bayesian Transportation Boundcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 6; 2025: 5; 2026: 1.

19 papers.

Sort:

By area:

  1. 2024Credal Learning Theorycited by 31
  2. 2023A unified recipe for deriving (time-uniform) PAC-Bayes boundscited by 25
  3. 2023Statistical Indistinguishability of Learning Algorithmscited by 17
  4. 2023More PAC-Bayes bounds: From bounded losses, to losses with general tail behaviors, to anytime validitycited by 17
  5. 2023PAC-Bayesian Generalization Bounds for Adversarial Generative Modelscited by 13
  6. 2024A PAC-Bayesian Link Between Generalisation and Flat Minimacited by 13
  7. 2023Learning via Wasserstein-Based High Probability Generalisation Boundscited by 12
  8. 2023Time-Independent Information-Theoretic Generalization Bounds for SGLDcited by 12
  9. 2024Neural Spacetimes for DAG Representation Learningcited by 5
  10. 2023Wasserstein PAC-Bayes Learning: Exploiting Optimisation Guarantees to Explain Generalisationcited by 3
  11. 2025Simultaneously Solving Infinitely Many LQ Mean Field Games In Hilbert Spaces: The Power of Neural Operatorscited by 3
  12. 2024Tighter Generalisation Bounds via Interpolationcited by 2
7 more
  1. 2024Multi-Group Proportional Representation in Retrievalcited by 2
  2. 2025Reconcile Certified Robustness and Accuracy for DNN-based Smoothed Majority Vote Classifiercited by 2
  3. 2024A Generalization Result for Convergence in Learning-to-Optimizecited by 1
  4. 2025PAC-Bayesian Reinforcement Learning Trains Generalizable Policiescited by 1
  5. 2025Stability-based Generalization Bounds for Variational Inferencecited by 0
  6. 2025Kernel Trace Distance: Quantum Statistical Metric between Measures through RKHS Density Operatorscited by 0
  7. 2026Testing Distributions Against Bounded Distinguisherscited by 0