Bound to Disagree: Generalization Bounds via Certifiable Surrogates
Not on the map. Why?
Abstract (arXiv)
Generalization bounds for deep learning models are typically vacuous, not computable or restricted to specific model classes. In this paper, we tackle these issues by providing new disagreement-based certificates for the gap between the true risk of any two predictors. We then bound the true risk of the predictor of interest via a surrogate model that enjoys tight generalization guarantees, and by evaluating our disagreement bound on an unlabeled dataset.We empirically demonstrate the tightness of the obtained certificates and showcase the versatility of the approach by training surrogate models leveraging three different frameworks: sample compression, model compression and PAC-Bayes theory. Importantly, such guarantees are achieved without modifying the target model, nor adapting the training procedure to the generalization framework.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 54 of its 106 references that are in the map. First posted on arXiv 26 Feb 2026.
Builds on
54 of 106 references.
References · 54
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2016Equality of Opportunity in Supervised Learningcited by 2,000
- 2016Understanding deep learning requires rethinking generalizationcited by 1,657
- 2017Spectrally-normalized margin bounds for neural networkscited by 681
- 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
- 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
- 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
- 2019Fantastic Generalization Measures and Where to Find Themcited by 507
38 more
- 2018Measuring the Intrinsic Dimension of Objective Landscapescited by 364
- 2018Stronger generalization bounds for deep nets via a compression approachcited by 306
- 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 304
- 2017Size-Independent Sample Complexity of Neural Networkscited by 289
- 2024The Road Less Scheduledcited by 158
- 2018Adversarial Active Learning for Deep Networks: a Margin Based Approachcited by 155
- 2022DeepCore: A Comprehensive Library for Coreset Selection in Deep Learningcited by 146
- 2020Reasoning About Generalization via Conditional Mutual Informationcited by 145
- 2018Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approachcited by 125
- 2020Tighter risk certificates for neural networkscited by 112
- 2017Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priorscited by 81
- 2022PAC-Bayes Compression Bounds So Tight That They Can Explain Generalizationcited by 75
- 2023Generalization Bounds: Perspectives from Information Theory and PAC-Bayescited by 60
- 2019Generalization bounds for deep convolutional neural networkscited by 58
- 2019Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margincited by 56
- 2023Non-Vacuous Generalization Bounds for Large Language Modelscited by 53
- 2020Generalization Bounds via Information Density and Conditional Information Densitycited by 43
- 2019Compression based bound for non-compressed network: unified generalization error analysis of large compressible deep neural networkcited by 39
- 2021Generalization bounds via distillationcited by 33
- 2022A New Family of Generalization Bounds Using Samplewise Evaluated CMIcited by 28
- 2025A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advancescited by 28
- 2024Unlocking Tokens as Data Points for Generalization Bounds on Larger Language Modelscited by 25
- 2025TorchAO: PyTorch-Native Training-to-Serving Model Optimizationcited by 25
- 2021How Tight Can PAC-Bayes be in the Small Data Regime?cited by 24
- 2019Generalization Bounds for Convolutional Neural Networkscited by 21
- 2024Mission Impossible: A Statistical Perspective on Jailbreaking LLMscited by 21
- 2022On PAC-Bayesian reconstruction guarantees for VAEscited by 20
- 2021Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasetscited by 17
- 2023Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theorycited by 16
- 2023PAC-Bayesian Generalization Bounds for Adversarial Generative Modelscited by 13
- 2024More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithmscited by 10
- 2023A Note on the Convergence of Denoising Diffusion Probabilistic Modelscited by 9
- 2021Self-Bounding Majority Vote Learning Algorithms by the Direct Minimization of a Tight PAC-Bayesian C-Boundcited by 8
- 2022A Note on the Chernoff Bound for Random Variables in the Unit Intervalcited by 7
- 2024Sample Compression Unleashed: New Generalization Bounds for Real Valued Lossescited by 4
- 2024On Generalization Bounds for Neural Networks with Low Rank Layerscited by 4
- 2025Pick-to-Learn for Systems and Control: Data-driven Synthesis with State-of-the-art Safety Guaranteescited by 4
- 2018On the Perceptron's Compressioncited by 3
50 most cited of 54 All 54 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.