# Second-Order Provable Defenses against Adversarial Attacks

arXiv: https://arxiv.org/abs/2006.00731 · HTML: https://mlsanity.com/paper/2006.00731/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2020.

## Built on by (most cited first, 30 of 40)

- [Certifying LLM Safety against Adversarial Prompting](https://mlsanity.com/paper/2309.02705/) (2023)
- [Skew Orthogonal Convolutions](https://mlsanity.com/paper/2105.11417/) (2021)
- [Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100](https://mlsanity.com/paper/2108.04062/) (2021)
- [Enhancing Adversarial Training with Second-Order Statistics of Weights](https://mlsanity.com/paper/2203.06020/) (2022)
- [Policy Smoothing for Provably Robust Reinforcement Learning](https://mlsanity.com/paper/2106.11420/) (2021)
- [On the Certified Robustness for Ensemble Models and Beyond](https://mlsanity.com/paper/2107.10873/) (2021)
- [Improved, Deterministic Smoothing for L_1 Certified Robustness](https://mlsanity.com/paper/2103.10834/) (2021)
- [TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothness](https://mlsanity.com/paper/2104.00671/) (2021)
- [CausalAdv: Adversarial Robustness through the Lens of Causality](https://mlsanity.com/paper/2106.06196/) (2021)
- [Robust Unlearnable Examples: Protecting Data Against Adversarial Learning](https://mlsanity.com/paper/2203.14533/) (2022)
- [Low Curvature Activations Reduce Overfitting in Adversarial Training](https://mlsanity.com/paper/2102.07861/) (2021)
- [Explicit Tradeoffs between Adversarial and Natural Distributional Robustness](https://mlsanity.com/paper/2209.07592/) (2022)
- [Efficiently Training Low-Curvature Neural Networks](https://mlsanity.com/paper/2206.07144/) (2022)
- [Center Smoothing: Certified Robustness for Networks with Structured Outputs](https://mlsanity.com/paper/2102.09701/) (2021)
- [Run-Off Election: Improved Provable Defense against Data Poisoning Attacks](https://mlsanity.com/paper/2302.02300/) (2023)
- [Improving Robustness by Enhancing Weak Subnets](https://mlsanity.com/paper/2201.12765/) (2022)
- [On the Role of Generalization in Transferability of Adversarial Examples](https://mlsanity.com/paper/2206.09238/) (2022)
- [Improved techniques for deterministic l2 robustness](https://mlsanity.com/paper/2211.08453/) (2022)
- [NoiLIn: Improving Adversarial Training and Correcting Stereotype of Noisy Labels](https://mlsanity.com/paper/2105.14676/) (2021)
- [Accelerating Certified Robustness Training via Knowledge Transfer](https://mlsanity.com/paper/2210.14283/) (2022)
- [Certification of Speaker Recognition Models to Additive Perturbations](https://mlsanity.com/paper/2404.18791/) (2024)
- [Certifying Model Accuracy under Distribution Shifts](https://mlsanity.com/paper/2201.12440/) (2022)
- [Neural Continuous-Time Supermartingale Certificates](https://mlsanity.com/paper/2412.17432/) (2024)
- [Explaining Adversarial Vulnerability with a Data Sparsity Hypothesis](https://mlsanity.com/paper/2103.00778/) (2021)
- [Compositional Curvature Bounds for Deep Neural Networks](https://mlsanity.com/paper/2406.05119/) (2024)
- [Provable Bounds on the Hessian of Neural Networks: Derivative-Preserving Reachability Analysis](https://mlsanity.com/paper/2406.04476/) (2024)
- [Structure-Aware Robustness Certificates for Graph Classification](https://mlsanity.com/paper/2306.11915/) (2023)
- [Exploring Geometry of Blind Spots in Vision Models](https://mlsanity.com/paper/2310.19889/) (2023)
- [Provable Robustness for Streaming Models with a Sliding Window](https://mlsanity.com/paper/2303.16308/) (2023)
- [Learning to Learn Transferable Generative Attack for Person Re-Identification](https://mlsanity.com/paper/2409.04208/) (2024)

## Abstract (arXiv)

A robustness certificate is the minimum distance of a given input to the decision boundary of the classifier (or its lower bound). For it any input perturbations with a magnitude smaller than the certificate value, the classification output will provably remain unchanged. Exactly computing the robustness certificates for neural networks is difficult since it requires solving a non-convex optimization. In this paper, we provide computationally-efficient robustness certificates for neural networks with differentiable activation functions in two steps. First, we show that if the eigenvalues of the Hessian of the network are bounded, we can compute a robustness certificate in the l2 norm efficiently using convex optimization. Second, we derive a computationally-efficient differentiable upper bound on the curvature of a deep network. We also use the curvature bound as a regularization term during the training of the network to boost its certified robustness. Putting these results together leads to our proposed bf Curvature-based bf Robustness bf Certificate (CRC) and bf Curvature-based bf Robust bf Training (CRT). Our numerical results show that CRT leads to significantly higher certified robust accuracy compared to interval-bound propagation (IBP) based training. We achieve certified robust accuracy 69.79%, 57.78% and 53.19% while IBP-based methods achieve 44.96%, 44.74% and 44.66% on 2,3 and 4 layer networks respectively on the MNIST-dataset.