# SOAR: Second-Order Adversarial Regularization

arXiv: https://arxiv.org/abs/2004.01832 · HTML: https://mlsanity.com/paper/2004.01832/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Apr 2020.

## Built on by (most cited first, 4 of 4)

- [Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression](https://mlsanity.com/paper/2109.03228/) (2021)
- [Learning Diverse-Structured Networks for Adversarial Robustness](https://mlsanity.com/paper/2102.01886/) (2021)
- [Bridging the Gap Between Adversarial Robustness and Optimization Bias](https://mlsanity.com/paper/2102.08868/) (2021)
- [SORA: Free Second-Order Attacks in Fast Adversarial Training](https://mlsanity.com/paper/2606.00738/) (2026)

## Abstract (arXiv)

Adversarial training is a common approach to improving the robustness of deep neural networks against adversarial examples. In this work, we propose a novel regularization approach as an alternative. To derive the regularizer, we formulate the adversarial robustness problem under the robust optimization framework and approximate the loss function using a second-order Taylor series expansion. Our proposed second-order adversarial regularizer (SOAR) is an upper bound based on the Taylor approximation of the inner-max in the robust optimization objective. We empirically show that the proposed method significantly improves the robustness of networks against the ℓ_∞ and ℓ2 bounded perturbations generated using cross-entropy-based PGD on CIFAR-10 and SVHN.