# Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations

arXiv: https://arxiv.org/abs/2002.04599 · HTML: https://mlsanity.com/paper/2002.04599/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Feb 2020.

## Built on by (most cited first, 30 of 55)

- [OR-Bench: An Over-Refusal Benchmark for Large Language Models](https://mlsanity.com/paper/2405.20947/) (2024)
- [Improving Robustness using Generated Data](https://mlsanity.com/paper/2110.09468/) (2021)
- [Partial success in closing the gap between human and machine vision](https://mlsanity.com/paper/2106.07411/) (2021)
- [Robustness and Accuracy Could Be Reconcilable by (Proper) Definition](https://mlsanity.com/paper/2202.10103/) (2022)
- [A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies](https://mlsanity.com/paper/2203.14046/) (2022)
- [Adversarial Robustness with Semi-Infinite Constrained Learning](https://mlsanity.com/paper/2110.15767/) (2021)
- [Exploring and Exploiting Decision Boundary Dynamics for Adversarial Robustness](https://mlsanity.com/paper/2302.03015/) (2023)
- [Attacking Adversarial Attacks as A Defense](https://mlsanity.com/paper/2106.04938/) (2021)
- [Adversarial Illusions in Multi-Modal Embeddings](https://mlsanity.com/paper/2308.11804/) (2023)
- [On the Limitations of Stochastic Pre-processing Defenses](https://mlsanity.com/paper/2206.09491/) (2022)
- [Scaling Adversarial Training to Large Perturbation Bounds](https://mlsanity.com/paper/2210.09852/) (2022)
- [CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models](https://mlsanity.com/paper/2501.14940/) (2025)
- [Revisiting Robustness in Graph Machine Learning](https://mlsanity.com/paper/2305.00851/) (2023)
- [ZeroGrad : Mitigating and Explaining Catastrophic Overfitting in FGSM Adversarial Training](https://mlsanity.com/paper/2103.15476/) (2021)
- [Towards Good Practices in Evaluating Transfer Adversarial Attacks](https://mlsanity.com/paper/2211.09565/) (2022)
- [Adversarial Visual Robustness by Causal Intervention](https://mlsanity.com/paper/2106.09534/) (2021)
- [Can Adversarial Training Be Manipulated By Non-Robust Features?](https://mlsanity.com/paper/2201.13329/) (2022)
- [Exposing Previously Undetectable Faults in Deep Neural Networks](https://mlsanity.com/paper/2106.00576/) (2021)
- [Multi-scale Diffusion Denoised Smoothing](https://mlsanity.com/paper/2310.16779/) (2023)
- [Invariance-Aware Randomized Smoothing Certificates](https://mlsanity.com/paper/2211.14207/) (2022)
- [Robustified ANNs Reveal Wormholes Between Human Category Percepts](https://mlsanity.com/paper/2308.06887/) (2023)
- [Assessing Robustness via Score-Based Adversarial Image Generation](https://mlsanity.com/paper/2310.04285/) (2023)
- [Understanding and Diagnosing Deep Reinforcement Learning](https://mlsanity.com/paper/2406.16979/) (2024)
- [Grid Cell Path Integration For Movement-Based Visual Object Recognition](https://mlsanity.com/paper/2102.09076/) (2021)
- [When Vision Fails: Text Attacks Against ViT and OCR](https://mlsanity.com/paper/2306.07033/) (2023)
- [Constrained Gradient Descent: A Powerful and Principled Evasion Attack Against Neural Networks](https://mlsanity.com/paper/2112.14232/) (2021)
- [Adversarially trained neural representations may already be as robust as corresponding biological neural representations](https://mlsanity.com/paper/2206.11228/) (2022)
- [Lagrangian Objective Function Leads to Improved Unforeseen Attack Generalization in Adversarial Training](https://mlsanity.com/paper/2103.15385/) (2021)
- [Balanced Adversarial Training: Balancing Tradeoffs between Fickleness and Obstinacy in NLP Models](https://mlsanity.com/paper/2210.11498/) (2022)
- [Certified Invertibility in Neural Networks via Mixed-Integer Programming](https://mlsanity.com/paper/2301.11783/) (2023)

## Abstract (arXiv)

Adversarial examples are malicious inputs crafted to induce misclassification. Commonly studied sensitivity-based adversarial examples introduce semantically-small changes to an input that result in a different model prediction. This paper studies a complementary failure mode, invariance-based adversarial examples, that introduce minimal semantic changes that modify an input's true label yet preserve the model's prediction. We demonstrate fundamental tradeoffs between these two types of adversarial examples. We show that defenses against sensitivity-based attacks actively harm a model's accuracy on invariance-based attacks, and that new approaches are needed to resist both attack types. In particular, we break state-of-the-art adversarially-trained and certifiably-robust models by generating small perturbations that the models are (provably) robust to, yet that change an input's class according to human labelers. Finally, we formally show that the existence of excessively invariant classifiers arises from the presence of overly-robust predictive features in standard datasets.