# Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness

arXiv: https://arxiv.org/abs/1903.10484 · HTML: https://mlsanity.com/paper/1903.10484/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2019.

## Built on by (most cited first, 9 of 9)

- [A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness](https://mlsanity.com/paper/2205.00403/) (2022)
- [A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies](https://mlsanity.com/paper/2203.14046/) (2022)
- [Privacy Inference Attacks and Defenses in Cloud-based Deep Neural Network: A Survey](https://mlsanity.com/paper/2105.06300/) (2021)
- [Federated and Transfer Learning: A Survey on Adversaries and Defense Mechanisms](https://mlsanity.com/paper/2207.02337/) (2022)
- [Balanced Adversarial Training: Balancing Tradeoffs between Fickleness and Obstinacy in NLP Models](https://mlsanity.com/paper/2210.11498/) (2022)
- [DiPSeN: Differentially Private Self-normalizing Neural Networks For Adversarial Robustness in Federated Learning](https://mlsanity.com/paper/2101.03218/) (2021)
- [On the Effect of Adversarial Training Against Invariance-based Adversarial Examples](https://mlsanity.com/paper/2302.08257/) (2023)
- [Adversarial Robustness Assessment of NeuroEvolution Approaches](https://mlsanity.com/paper/2207.05451/) (2022)
- [Training Image Derivatives: Increased Accuracy and Universal Robustness](https://mlsanity.com/paper/2310.14045/) (2023)

## Abstract (arXiv)

Adversarial examples are malicious inputs crafted to cause a model to misclassify them. Their most common instantiation, "perturbation-based" adversarial examples introduce changes to the input that leave its true label unchanged, yet result in a different model prediction. Conversely, "invariance-based" adversarial examples insert changes to the input that leave the model's prediction unaffected despite the underlying input's label having changed. In this paper, we demonstrate that robustness to perturbation-based adversarial examples is not only insufficient for general robustness, but worse, it can also increase vulnerability of the model to invariance-based adversarial examples. In addition to analytical constructions, we empirically study vision classifiers with state-of-the-art robustness to perturbation-based adversaries constrained by an ℓp norm. We mount attacks that exploit excessive model invariance in directions relevant to the task, which are able to find adversarial examples within the ℓp ball. In fact, we find that classifiers trained to be ℓp-norm robust are more vulnerable to invariance-based adversarial examples than their undefended counterparts. Excessive invariance is not limited to models trained to be robust to perturbation-based ℓp-norm adversaries. In fact, we argue that the term adversarial example is used to capture a series of model limitations, some of which may not have been discovered yet. Accordingly, we call for a set of precise definitions that taxonomize and address each of these shortcomings in learning.