# Certified Patch Robustness via Smoothed Vision Transformers

arXiv: https://arxiv.org/abs/2110.07719 · HTML: https://mlsanity.com/paper/2110.07719/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 64 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks](https://mlsanity.com/paper/1702.01135/) (2017)
- [Provable defenses against adversarial examples via the convex outer adversarial polytope](https://mlsanity.com/paper/1711.00851/) (2017)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Adversarial Patch](https://mlsanity.com/paper/1712.09665/) (2017)
- [Certified Robustness to Adversarial Examples with Differential Privacy](https://mlsanity.com/paper/1802.03471/) (2018)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [Efficient Neural Network Robustness Certification with General Activation Functions](https://mlsanity.com/paper/1811.00866/) (2018)
- [Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers](https://mlsanity.com/paper/1906.04584/) (2019)
- [On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models](https://mlsanity.com/paper/1810.12715/) (2018)
- [Certified Defenses against Adversarial Examples](https://mlsanity.com/paper/1801.09344/) (2018)
- [Fooling automated surveillance cameras: adversarial patches to attack person detection](https://mlsanity.com/paper/1904.08653/) (2019)
- [Towards Fast Computation of Certified Robustness for ReLU Networks](https://mlsanity.com/paper/1804.09699/) (2018)
- [Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks](https://mlsanity.com/paper/1705.01320/) (2017)
- [Certified Adversarial Robustness with Additive Noise](https://mlsanity.com/paper/1809.03113/) (2018)
- [Physical Adversarial Examples for Object Detectors](https://mlsanity.com/paper/1807.07769/) (2018)
- [ShapeShifter: Robust Physical Adversarial Attack on Faster R-CNN Object Detector](https://mlsanity.com/paper/1804.05810/) (2018)
- [Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors](https://mlsanity.com/paper/1910.14667/) (2019)
- [DPatch: An Adversarial Patch Attack on Object Detectors](https://mlsanity.com/paper/1806.02299/) (2018)
- [Randomized Smoothing of All Shapes and Sizes](https://mlsanity.com/paper/2002.08118/) (2020)
- [Scaling provable adversarial defenses](https://mlsanity.com/paper/1805.12514/) (2018)
- [An approach to reachability analysis for feed-forward ReLU neural networks](https://mlsanity.com/paper/1706.07351/) (2017)

## Built on by (most cited first, 30 of 45)

- [How Deep Learning Sees the World: A Survey on Adversarial Attacks & Defenses](https://mlsanity.com/paper/2305.10862/) (2023)
- [Are Vision Transformers Robust to Patch Perturbations?](https://mlsanity.com/paper/2111.10659/) (2021)
- [How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective](https://mlsanity.com/paper/2203.14195/) (2022)
- [Missingness Bias in Model Debugging](https://mlsanity.com/paper/2204.08945/) (2022)
- [Jedi: Entropy-based Localization and Removal of Adversarial Patches](https://mlsanity.com/paper/2304.10029/) (2023)
- [Towards Efficient Adversarial Training on Vision Transformers](https://mlsanity.com/paper/2207.10498/) (2022)
- [ObjectSeeker: Certifiably Robust Object Detection against Patch Hiding Attacks via Patch-agnostic Masking](https://mlsanity.com/paper/2202.01811/) (2022)
- [Estimating Conditional Mutual Information for Dynamic Feature Selection](https://mlsanity.com/paper/2306.03301/) (2023)
- [Adversarial Patch Attacks and Defences in Vision-Based Tasks: A Survey](https://mlsanity.com/paper/2206.08304/) (2022)
- [Certified Defences Against Adversarial Patch Attacks on Semantic Segmentation](https://mlsanity.com/paper/2209.05980/) (2022)
- [Stability Guarantees for Feature Attributions with Multiplicative Smoothing](https://mlsanity.com/paper/2307.05902/) (2023)
- [CTBENCH: A Library and Benchmark for Certified Training](https://mlsanity.com/paper/2406.04848/) (2024)
- [PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing](https://mlsanity.com/paper/2111.10481/) (2021)
- [Improving Interpretation Faithfulness for Vision Transformers](https://mlsanity.com/paper/2311.17983/) (2023)
- [Visual Adversarial Attacks and Defenses in the Physical World: A Survey](https://mlsanity.com/paper/2211.01671/) (2022)
- [Certified Zeroth-order Black-Box Defense with Robust UNet Denoiser](https://mlsanity.com/paper/2304.06430/) (2023)
- [PatchCURE: Improving Certifiable Robustness, Model Utility, and Computation Efficiency of Adversarial Patch Defenses](https://mlsanity.com/paper/2310.13076/) (2023)
- [Improved techniques for deterministic l2 robustness](https://mlsanity.com/paper/2211.08453/) (2022)
- [Confidence-aware Training of Smoothed Classifiers for Certified Robustness](https://mlsanity.com/paper/2212.09000/) (2022)
- [Towards Improved Input Masking for Convolutional Neural Networks](https://mlsanity.com/paper/2211.14646/) (2022)
- [Benchmarking Robustness to Adversarial Image Obfuscations](https://mlsanity.com/paper/2301.12993/) (2023)
- [Intriguing Equivalence Structures of the Embedding Space of Vision Transformers](https://mlsanity.com/paper/2401.15568/) (2024)
- [Fairness-aware Vision Transformer via Debiased Self-Attention](https://mlsanity.com/paper/2301.13803/) (2023)
- [Revisiting Image Classifier Training for Improved Certified Robust Defense against Adversarial Patches](https://mlsanity.com/paper/2306.12610/) (2023)
- [CrossCert: A Cross-Checking Detection Approach to Patch Robustness Certification for Deep Learning Models](https://mlsanity.com/paper/2405.07668/) (2024)
- [Stable Vision Concept Transformers for Medical Diagnosis](https://mlsanity.com/paper/2506.05286/) (2025)
- [Random Position Adversarial Patch for Vision Transformers](https://mlsanity.com/paper/2307.04066/) (2023)
- [A Majority Invariant Approach to Patch Robustness Certification for Deep Learning Models](https://mlsanity.com/paper/2308.00452/) (2023)
- [Towards Robust Semantic Segmentation against Patch-based Attack via Attention Refinement](https://mlsanity.com/paper/2401.01750/) (2024)
- [Machine unlearning through fine-grained model parameters perturbation](https://mlsanity.com/paper/2401.04385/) (2024)

## Abstract (arXiv)

Certified patch defenses can guarantee robustness of an image classifier to arbitrary changes within a bounded contiguous region. But, currently, this robustness comes at a cost of degraded standard accuracies and slower inference times. We demonstrate how using vision transformers enables significantly better certified patch robustness that is also more computationally efficient and does not incur a substantial drop in standard accuracy. These improvements stem from the inherent ability of the vision transformer to gracefully handle largely masked images. Our code is available at https://github.com/MadryLab/smoothed-vit.