Simpler Certified Radius Maximization by Propagating Covariances
Not on the map. Why?
Abstract (arXiv)
One strategy for adversarially training a robust model is to maximize its certified radius -- the neighborhood around a given training sample for which the model's prediction remains unchanged. The scheme typically involves analyzing a "smoothed" classifier where one estimates the prediction corresponding to Gaussian samples in the neighborhood of each sample in the mini-batch, accomplished in practice by Monte Carlo sampling. In this paper, we investigate the hypothesis that this sampling bottleneck can potentially be mitigated by identifying ways to directly propagate the covariance matrix of the smoothed distribution through the network. To this end, we find that other than certain adjustments to the network, propagating the covariances must also be accompanied by additional accounting that keeps track of how the distributional moments transform and interact at each stage in the network. We show how satisfying these criteria yields an algorithm for maximizing the certified radius on datasets including Cifar-10, ImageNet, and Places365 while offering runtime savings on networks with moderate depth, with a small compromise in overall accuracy. We describe the details of the key modifications that enable practical use. Via various experiments, we evaluate when our simplifications are sensible, and what the key benefits and limitations are.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 42 of its 63 references that are in the map. First posted on arXiv 13 Apr 2021.
Builds on
42 of 63 references.
References · 42
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2016Layer Normalizationcited by 5,579
- 2013Intriguing properties of neural networkscited by 5,114
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2016Identity Mappings in Deep Residual Networkscited by 1,960
- 2016Understanding deep learning requires rethinking generalizationcited by 1,657
- 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
- 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
30 more
- 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
- 2016End to End Learning for Self-Driving Carscited by 1,000
- 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
- 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
- 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
- 2017Evasion Attacks against Machine Learning at Test Timecited by 686
- 2018Learning to Reweight Examples for Robust Deep Learningcited by 670
- 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
- 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
- 2019Adversarial Training for Free!cited by 638
- 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
- 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
- 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 499
- 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 462
- 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 439
- 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
- 2014Training Deep Neural Networks on Noisy Labels with Bootstrappingcited by 322
- 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
- 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 254
- 2017Decoupling "when to update" from "how to update"cited by 230
- 2018Adversarial Risk and the Dangers of Evaluating Against Weak Attackscited by 221
- 2019You Only Propagate Once: Accelerating Adversarial Training via Maximal Principlecited by 181
- 2018Certified Adversarial Robustness with Additive Noisecited by 178
- 2018Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networkscited by 176
- 2020MACER: Attack-free and Scalable Robust Training via Maximizing Certified Radiuscited by 114
- 2015Learning with a Strong Adversarycited by 68
- 2018On the Robustness of the CVPR 2018 White-Box Adversarial Example Defensescited by 40
- 2019Generating Accurate Pseudo-labels in Semi-Supervised Learning and Avoiding Overconfident Predictions via Hermite Polynomial Activationscited by 13
- 2019ProbAct: A Probabilistic Activation Function for Deep Neural Networkscited by 12
- 2020Towards an Adversarially Robust Normalization Approachcited by 9
Built on by
Citing papers per year: 2021: 1; 2022: 1.
2 papers.