mlsanityData to 7 Oct 2026

CertViT: Certified Robustness of Pre-Trained Vision Transformers

arXiv 2302.10287 ↗First posted 1 Feb 2023cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Lipschitz bounded neural networks are certifiably robust and have a good trade-off between clean and certified accuracy. Existing Lipschitz bounding methods train from scratch and are limited to moderately sized networks (< 6M parameters). They require a fair amount of hyper-parameter tuning and are computationally prohibitive for large networks like Vision Transformers (5M to 660M parameters). Obtaining certified robustness of transformers is not feasible due to the non-scalability and inflexibility of the current methods. This work presents CertViT, a two-step proximal-projection method to achieve certified robustness from pre-trained weights. The proximal step tries to lower the Lipschitz bound and the projection step tries to maintain the clean accuracy of pre-trained weights. We show that CertViT networks have better certified accuracy than state-of-the-art Lipschitz trained networks. We apply CertViT on several variants of pre-trained vision transformers and show adversarial robustness using standard attacks. Code : https://github.com/sagarverma/transformer-lipschitz

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 1 Feb 2023.

Builds on

27 of 37 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  7. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  8. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  9. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  10. 2018Regularisation of Neural Networks by Enforcing Lipschitz Continuitycited by 311
  11. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  12. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
15 more
  1. 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 254
  2. 2021Understanding Robustness of Transformers for Image Classificationcited by 250
  3. 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 204
  4. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 193
  5. 2020The Lipschitz Constant of Self-Attentioncited by 171
  6. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  7. 2021Globally-Robust Neural Networkscited by 106
  8. 2020Exactly Computing the Local Lipschitz Constant of ReLU Networkscited by 102
  9. 2020Lipschitz constant estimation of Neural Networks via sparse polynomial optimizationcited by 87
  10. 2021Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Boundscited by 64
  11. 2021Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNscited by 57
  12. 2019Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operatorscited by 54
  13. 2018RecurJac: An Efficient Recursive Algorithm for Bounding Jacobian Matrix of Neural Networks and Its Applicationscited by 27
  14. 2020Achieving robustness in classification using optimal transport with hinge regularizationcited by 27
  15. 2018Deep Neural Network Structures Solving Variational Inequalitiescited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 0; 2025: 2; 2026: 2.

6 papers.

  1. 2023How Smooth Is Attention?cited by 44
  2. 2023Mitigating Transformer Overconfidence via Lipschitz Regularizationcited by 15
  3. 2025Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliabilitycited by 6
  4. 2026Approximation Theory for Lipschitz Continuous Transformerscited by 5
  5. 2025LipShiFT: A Certifiably Robust Shift-based Vision Transformercited by 3
  6. 2026Architecture-agnostic Lipschitz-constant Bayesian header and its application to resolve semantically proximal classification errors with vision transformerscited by 0