mlsanityData to 14 Oct 2026

Accelerating Certified Robustness Training via Knowledge Transfer

arXiv 2210.14283 ↗First posted 25 Oct 2022cs.LG cs.CR cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

Training deep neural network classifiers that are certifiably robust against adversarial attacks is critical to ensuring the security and reliability of AI-controlled systems. Although numerous state-of-the-art certified training methods have been developed, they are computationally expensive and scale poorly with respect to both dataset and network complexity. Widespread usage of certified training is further hindered by the fact that periodic retraining is necessary to incorporate new data and network improvements. In this paper, we propose Certified Robustness Transfer (CRT), a general-purpose framework for reducing the computational overhead of any certifiably robust training method through knowledge transfer. Given a robust teacher, our framework uses a novel training loss to transfer the teacher's robustness to the student. We provide theoretical and empirical validation of CRT. Our experiments on CIFAR-10 show that CRT speeds up certified robustness training by 8 × on average across three different architecture generations while achieving comparable robustness to state-of-the-art methods. We also show that CRT can scale to large-scale datasets like ImageNet.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 37 of its 53 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

37 of 53 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  7. 2013Intriguing properties of neural networkscited by 5,155
  8. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  10. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  12. 2020Designing Network Design Spacescited by 960
25 more
  1. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  2. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 744
  3. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 569
  4. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
  5. 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
  6. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 468
  7. 2017Deep Layer Aggregationcited by 436
  8. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 330
  9. 2018Feature Denoising for Improving Adversarial Robustnesscited by 314
  10. 2018Certified Defenses against Adversarial Examplescited by 306
  11. 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 205
  12. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 185
  13. 2018Certified Adversarial Robustness with Additive Noisecited by 180
  14. 2020Randomized Smoothing of All Shapes and Sizescited by 147
  15. 2018Scaling provable adversarial defensescited by 143
  16. 2019Adversarially Robust Distillationcited by 125
  17. 2020MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radiuscited by 114
  18. 2020SoK: Certified Robustness for Deep Neural Networkscited by 105
  19. 2021Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Bettercited by 79
  20. 2020Consistency Regularization for Certified Robustness of Smoothed Classifierscited by 73
  21. 2019Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifierscited by 65
  22. 2021Reliable Adversarial Distillation with Unreliable Teacherscited by 56
  23. 2021SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustnesscited by 49
  24. 2020Second-Order Provable Defenses against Adversarial Attackscited by 40
  25. 2019What it Thinks is Important is Important: Robustness Transfers through Input Gradientscited by 34

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 4.

7 papers.

  1. 2024Certified Robust Accuracy of Neural Networks Are Bounded due to Bayes Errorscited by 7
  2. 2024Gaussian Loss Smoothing Enables Certified Training with Tight Convex Relaxationscited by 4
  3. 2023Rethinking Randomized Smoothing from the Perspective of Scalabilitycited by 3
  4. 2022Multi-head Ensemble of Smoothed Classifiers for Certified Robustnesscited by 2
  5. 2023Towards Certified Probabilistic Robustness with High Accuracycited by 2
  6. 2024Average Certified Radius is a Poor Metric for Randomized Smoothingcited by 1
  7. 2024Optimal Zero-Shot Detector for Multi-Armed Attackscited by 0