Certified Robust Models with Slack Control and Large Lipschitz Constants
Not on the map. Why?
Abstract (arXiv)
Despite recent success, state-of-the-art learning-based models remain highly vulnerable to input changes such as adversarial examples. In order to obtain certifiable robustness against such perturbations, recent work considers Lipschitz-based regularizers or constraints while at the same time increasing prediction margin. Unfortunately, this comes at the cost of significantly decreased accuracy. In this paper, we propose a Calibrated Lipschitz-Margin Loss (CLL) that addresses this issue and improves certified robustness by tackling two problems: Firstly, commonly used margin losses do not adjust the penalties to the shrinking output distribution; caused by minimizing the Lipschitz constant K. Secondly, and most importantly, we observe that minimization of K can lead to overly smooth decision functions. This limits the model's complexity and thus reduces accuracy. Our CLL addresses these issues by explicitly calibrating the loss w.r.t. margin and Lipschitz constant, thereby establishing full control over slack and improving robustness certificates even with larger Lipschitz constants. On CIFAR-10, CIFAR-100 and Tiny-ImageNet, our models consistently outperform losses that leave the constant unattended. On CIFAR-100 and Tiny-ImageNet, CLL improves upon state-of-the-art deterministic L2 robust accuracies. In contrast to current trends, we unlock potential of much smaller models without K=1 constraints.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 42 of its 54 references that are in the map. First posted on arXiv 12 Sep 2023.
Builds on
42 of 54 references.
References · 42
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
- 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
- 2017Spectrally-normalized margin bounds for neural networkscited by 681
- 2019On Evaluating Adversarial Robustnesscited by 442
- 2019Unlabeled Data Improves Adversarial Robustnesscited by 415
- 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
- 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
- 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
- 2018Sorting out Lipschitz function approximationcited by 211
30 more
- 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 204
- 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 193
- 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 182
- 2018MMA Training: Direct Input Space Margin Maximization through Adversarial Trainingcited by 171
- 2014Deep Learning Face Representation by Joint Identification-Verificationcited by 162
- 2018Scaling provable adversarial defensescited by 141
- 2019Robust Learning with Jacobian Regularizationcited by 127
- 2016Robust Large Margin Deep Neural Networkscited by 117
- 2018Large Margin Deep Networks for Classificationcited by 110
- 2021Globally-Robust Neural Networkscited by 106
- 2021Orthogonalizing Convolutional Layers with the Cayley Transformcited by 105
- 2020Exactly Computing the Local Lipschitz Constant of ReLU Networkscited by 102
- 2020SoK: Certified Robustness for Deep Neural Networkscited by 100
- 2018Improving DNN Robustness to Adversarial Attacks using Jacobian Regularizationcited by 88
- 2019Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networkscited by 86
- 2021Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Boundscited by 64
- 2021A Dynamical System Perspective for Lipschitz Neural Networkscited by 59
- 2023A Unified Algebraic Perspective on Lipschitz Neural Networkscited by 58
- 2019Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margincited by 56
- 2023Direct Parameterization of Lipschitz-Bounded Deep Networkscited by 49
- 2018Limitations of the Lipschitz constant as a defense against adversarial examplescited by 48
- 2021Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100cited by 47
- 2020A case for new neural network smoothness constraintscited by 43
- 2021Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neuronscited by 40
- 2022Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networkscited by 39
- 2022LOT: Layer-wise Orthogonal Training on Improving ℓ2 Certified Robustnesscited by 32
- 2021Boosting the Certified Robustness of L-infinity Distance Netscited by 24
- 2023Efficient Bound of Lipschitz Constant for Convolutional Layers by Gram Iterationcited by 19
- 2021Recent Advances in Large Margin Learningcited by 7
- 2015On the Depth of Deep Neural Networks: A Theoretical Viewcited by 3
Built on by
Citing papers per year: 2023: 1; 2024: 2.
3 papers.