mlsanityData to 7 Oct 2026

Formalizing Generalization and Robustness of Neural Networks to Weight Perturbations

arXiv 2103.02200 ↗First posted 3 Mar 2021cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

Studying the sensitivity of weight perturbation in neural networks and its impacts on model performance, including generalization and robustness, is an active research topic due to its implications on a wide range of machine learning tasks such as model compression, generalization gap assessment, and adversarial attacks. In this paper, we provide the first integral study and analysis for feed-forward neural networks in terms of the robustness in pairwise class margin and its generalization behavior under weight perturbation. We further design a new theory-driven loss function for training generalizable and robust neural networks against weight perturbations. Empirical experiments are conducted to validate our theoretical analysis. Our results offer fundamental insights for characterizing the generalization and robustness of neural networks against weight perturbations.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 19 of its 39 references that are in the map. First posted on arXiv 3 Mar 2021.

Builds on

19 of 39 references.

References · 19

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  3. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  4. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  5. 2018Robustness May Be at Odds with Accuracycited by 873
  6. 2017Exploring Generalization in Deep Learningcited by 595
  7. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  8. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  9. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  10. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  11. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  12. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 223
7 more
  1. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  2. 2018Rademacher Complexity for Adversarially Robust Generalizationcited by 132
  3. 2020Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustnesscited by 130
  4. 2019Fault Sneaking Attack: a Stealthy Framework for Misleading Deep Neural Networkscited by 25
  5. 2018Approximation and Estimation for High-Dimensional Deep Learning Networkscited by 22
  6. 2019Diametrical Risk Minimization: Theory and Computationscited by 15
  7. 2019Global Capacity Measures for Deep ReLU Networks via Path Samplingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 5; 2024: 4; 2025: 5; 2026: 3.

22 papers.

Sort:

By area:

  1. 2022Rashomon Capacity: A Metric for Predictive Multiplicity in Classificationcited by 46
  2. 2022Formalising the Robustness of Counterfactual Explanations for Neural Networkscited by 24
  3. 2022Computing-In-Memory Neural Network Accelerators for Safety-Critical Systems: Can Small Device Variations Be Disastrous?cited by 12
  4. 2022Holistic Adversarial Robustness of Deep Learning Modelscited by 11
  5. 2022A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunitiescited by 8
  6. 2023Improving Realistic Worst-Case Performance of NVCiM DNN Accelerators through Training with Right-Censored Gaussian Noisecited by 8
  7. 2024DPAdapter: Improving Differentially Private Deep Learning through Noise Tolerance Pre-trainingcited by 7
  8. 2024Interval Abstractions for Robust Counterfactual Explanationscited by 7
  9. 2025StealthAttack: Robust 3D Gaussian Splatting Poisoning via Density-Guided Illusionscited by 6
  10. 2023Exploring the cloud of feature interaction scores in a Rashomon setcited by 5
  11. 2023Investigating Weight-Perturbed Deep Neural Networks With Application in Iris Presentation Attack Detectioncited by 3
  12. 2024Practical Attribution Guidance for Rashomon Setscited by 3
10 more
  1. 2023Compute-in-Memory based Neural Network Accelerators for Safety-Critical Systems: Worst-Case Scenarios and Protectionscited by 2
  2. 2025Adversarial Defence without Adversarial Defence: Enhancing Language Model Robustness via Instance-level Principal Component Removalcited by 2
  3. 2025Enhancing Offline Model-Based RL via Active Model Selection: A Bayesian Optimization Perspectivecited by 1
  4. 2026A2SG:Adaptive and Asymmetric Surrogate Gradients for Training Deep Spiking Neural Networkscited by 1
  5. 2023Verification of Neural Networks Local Differential Classification Privacycited by 0
  6. 2024Enhancing selectivity using Wasserstein distance based reweighingcited by 0
  7. 2025Beyond topography: Topographic regularization improves robustness and reshapes representations in convolutional neural networkscited by 0
  8. 2025MEASER: Malware embedding attacks on open-source LLMscited by 0
  9. 2026NNV3: Expanding Neural Network Verification to New Architectures and Domainscited by 0
  10. 2026Security Properties of Neural Networks as Decision Problemscited by 0