mlsanityData to 7 Oct 2026

On Rademacher Complexity-based Generalization Bounds for Deep Learning

arXiv 2208.04284 ↗First posted 8 Aug 2022stat.ML cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

We show that the Rademacher complexity-based framework can establish non-vacuous generalization bounds for Convolutional Neural Networks (CNNs) in the context of classifying a small set of image classes. A key technical advancement is the formulation of novel contraction lemmas for high-dimensional mappings between vector spaces, specifically designed for general Lipschitz activation functions. These lemmas extend and refine the Talagrand contraction lemma across a broader range of scenarios. Our Rademacher complexity bound provides an enhancement over the results presented by Golowich et al. for ReLU-based Deep Neural Networks (DNNs). Moreover, while previous works utilizing Rademacher complexity have primarily focused on ReLU DNNs, our results generalize to a wider class of activation functions.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 17 of its 42 references that are in the map. First posted on arXiv 8 Aug 2022.

Builds on

17 of 42 references.

References · 17

  1. 2017Spectrally-normalized margin bounds for neural networkscited by 681
  2. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  3. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  4. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  5. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 299
  6. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  7. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 223
  8. 2018Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approachcited by 125
  9. 2019Generalization Error Bounds Via R'enyi-, f-Divergences and Maximal Leakagecited by 48
  10. 2018Generalization Error in Deep Learningcited by 46
  11. 2020Differentiable PAC-Bayes Objectives with Partially Aggregated Neural Networkscited by 29
  12. 2019Estimating the Mixing Time of Ergodic Markov Chainscited by 24
5 more
  1. 2013Learning Theory and Algorithms for Revenue Optimization in Second-Price Auctions with Reservecited by 16
  2. 2019A Multistep Lyapunov Approach for Finite-Time Analysis of Biased Stochastic Approximationcited by 10
  3. 2021Wide stochastic networks: Gaussian limit and PAC-Bayesian trainingcited by 10
  4. 2021Generalization Error Bounds on Deep Learning with Markov Datasetscited by 9
  5. 2022Generalization Bounds on Multi-Kernel Learning with Mixed Datasetscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 5; 2025: 0; 2026: 6.

16 papers.

Sort:

By area:

  1. 2023How Does Information Bottleneck Help Deep Learning?cited by 92
  2. 2023Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equationcited by 32
  3. 2023Sequence Length Independent Norm-Based Generalization Bounds for Transformerscited by 28
  4. 2024How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learningcited by 6
  5. 2023Global Convergence Rate of Deep Equilibrium Models with General Activationscited by 4
  6. 2024On Generalisation Error Bounds for Transformerscited by 3
  7. 2024Length independent generalization bounds for deep SSM architectures via Rademacher contraction and stability constraintscited by 2
  8. 2024On Discriminative Probabilistic Modeling for Self-Supervised Representation Learningcited by 2
  9. 2026Process-Guided Concept Bottleneck Modelcited by 2
  10. 2023A Statistical Guarantee for Representation Transfer in Multitask Imitation Learningcited by 1
  11. 2024Multi-View Majority Vote Learning Algorithms: Direct Minimization of PAC-Bayesian Boundscited by 1
  12. 2026Lossy Common Information in a Learnable Gray-Wyner Networkcited by 1
4 more
  1. 2026Understanding Generalization from Embedding Dimension and Distributional Convergencecited by 1
  2. 2026Transfer Learning Through Conditional Quantile Matchingcited by 1
  3. 2026PAC-Bayes Bounds for Gibbs Posteriors via Singular Learning Theorycited by 0
  4. 2026A Rigorous, Tractable Measure of Model Complexitycited by 0