mlsanityData to 7 Oct 2026

Unlocking Deterministic Robustness Certification on ImageNet

arXiv 2301.12549 ↗First posted 29 Jan 2023cs.LG cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Despite the promise of Lipschitz-based methods for provably-robust deep learning with deterministic guarantees, current state-of-the-art results are limited to feed-forward Convolutional Networks (ConvNets) on low-dimensional data, such as CIFAR-10. This paper investigates strategies for expanding certifiably robust training to larger, deeper models. A key challenge in certifying deep networks is efficient calculation of the Lipschitz bound for residual blocks found in ResNet and ViT architectures. We show that fast ways of bounding the Lipschitz constant for conventional ResNets are loose, and show how to address this by designing a new residual block, leading to the Linear ResNet (LiResNet) architecture. We then introduce Efficient Margin MAximization (EMMA), a loss function that stabilizes robust training by simultaneously penalizing worst-case adversarial examples from all classes. Together, these contributions yield new state-of-the-art robust accuracy on CIFAR-10/100 and Tiny-ImageNet under ℓ2 perturbations. Moreover, for the first time, we are able to scale up fast deterministic robustness guarantees to ImageNet, demonstrating that this approach to robust learning can be applied to real-world applications. We release our code on Github: urlhttps://github.com/klasleino/gloro.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 52 of its 61 references that are in the map. First posted on arXiv 29 Jan 2023.

Builds on

52 of 61 references.

References · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  10. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  11. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  12. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
38 more
  1. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  2. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  3. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  4. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  5. 2021MetaFormer Is Actually What You Need for Visioncited by 523
  6. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
  7. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 388
  8. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  9. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 333
  10. 2020Are we done with ImageNet?cited by 329
  11. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 328
  12. 2021Improving Robustness using Generated Datacited by 233
  13. 2018Sorting out Lipschitz function approximationcited by 211
  14. 2019Fixup Initialization: Residual Learning Without Normalizationcited by 194
  15. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 193
  16. 2019ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blockscited by 178
  17. 2020The Lipschitz Constant of Self-Attentioncited by 171
  18. 2021A Universal Law of Robustness via Isoperimetrycited by 150
  19. 2022(Certified!!) Adversarial Robustness for Free!cited by 130
  20. 2021Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labelscited by 117
  21. 2021Globally-Robust Neural Networkscited by 106
  22. 2021Orthogonalizing Convolutional Layers with the Cayley Transformcited by 105
  23. 2019Preventing Gradient Attenuation in Lipschitz Constrained Convolutional Networkscited by 86
  24. 2018Generalizable Adversarial Training via Spectral Normalizationcited by 68
  25. 2021Skew Orthogonal Convolutionscited by 67
  26. 2021Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Boundscited by 64
  27. 2022When does dough become a bagel? Analyzing the remaining mistakes on ImageNetcited by 60
  28. 2021A Dynamical System Perspective for Lipschitz Neural Networkscited by 59
  29. 2018Provable Robustness of ReLU networks via Maximization of Linear Regionscited by 51
  30. 2021SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustnesscited by 49
  31. 2017DiracNets: Training Very Deep Neural Networks Without Skip-Connectionscited by 47
  32. 2021Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100cited by 47
  33. 2021Lipschitz Normalization for Self-Attention Layers with Application to Graph Neural Networkscited by 38
  34. 2021Scalable Certified Segmentation via Randomized Smoothingcited by 35
  35. 2021RepMLPNet: Hierarchical Vision MLP with Re-parameterized Localitycited by 32
  36. 2022LOT: Layer-wise Orthogonal Training on Improving ℓ2 Certified Robustnesscited by 32
  37. 2019Provable Certificates for Adversarial Examples: Fitting a Ball in the Union of Polytopescited by 25
  38. 2020Fast Geometric Projections for Local Robustness Certificationcited by 21

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 3; 2025: 3; 2026: 1.

13 papers.

Sort:

By area:

  1. 20231-Lipschitz Layers Compared: Memory, Speed, and Certifiable Robustnesscited by 15
  2. 2025SDP-CROWN: Efficient Bound Propagation for Neural Network Verification with Tightness of Semidefinite Programmingcited by 10
  3. 2023Raising the Bar for Certified Adversarial Robustness with Diffusion Modelscited by 7
  4. 2024Novel Quadratic Constraints for Extending LipSDP beyond Slope-Restricted Activationscited by 6
  5. 2025Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Losscited by 5
  6. 2024Intriguing Properties of Robust Classificationcited by 3
  7. 2023VNN: Verification-Friendly Neural Networks with Hard Robustness Guaranteescited by 2
  8. 2025Bridging the Theoretical Gap in Randomized Smoothingcited by 2
  9. 2023Is Certifying ℓp Robustness Still Worthwhile?cited by 1
  10. 2023Tailoring Adversarial Attacks on Deep Neural Networks for Targeted Class Manipulation Using DeepFool Algorithmcited by 1
  11. 2024Towards Generalized Certified Robustness with Multi-Norm Trainingcited by 1
  12. 2026Learning Better Certified Models from Empirically-Robust Teacherscited by 1
1 more
  1. 2023Robust and Efficient Interference Neural Networks for Defending Against Adversarial Attacks in ImageNetcited by 0