# Automated Discovery of Adaptive Attacks on Adversarial Defenses

arXiv: https://arxiv.org/abs/2102.11860 · HTML: https://mlsanity.com/paper/2102.11860/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 51 references that are in the map.
- First posted on arXiv 23 Feb 2021.

## Builds on (most cited first, 30 of 36)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods](https://mlsanity.com/paper/1705.07263/) (2017)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [Unlabeled Data Improves Adversarial Robustness](https://mlsanity.com/paper/1905.13736/) (2019)
- [Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One](https://mlsanity.com/paper/1912.03263/) (2019)
- [Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack](https://mlsanity.com/paper/1907.02044/) (2019)
- [Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples](https://mlsanity.com/paper/2010.03593/) (2020)
- [On Detecting Adversarial Perturbations](https://mlsanity.com/paper/1702.04267/) (2017)
- [Non-stochastic Best Arm Identification and Hyperparameter Optimization](https://mlsanity.com/paper/1502.07943/) (2015)
- [Improving Adversarial Robustness via Promoting Ensemble Diversity](https://mlsanity.com/paper/1901.08846/) (2019)
- [Geometry-aware Instance-reweighted Adversarial Training](https://mlsanity.com/paper/2010.01736/) (2020)
- [Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training](https://mlsanity.com/paper/1907.10764/) (2019)
- [Keeping the Bad Guys Out: Protecting and Vaccinating Deep Learning with JPEG Compression](https://mlsanity.com/paper/1705.02900/) (2017)
- [Adversarial Examples Detection in Deep Networks with Convolutional Filter Statistics](https://mlsanity.com/paper/1612.07767/) (2016)
- [Adversarial Manipulation of Deep Representations](https://mlsanity.com/paper/1511.05122/) (2015)

## Built on by (most cited first, 22 of 22)

- [DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purification](https://mlsanity.com/paper/2311.16124/) (2023)
- [Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples](https://mlsanity.com/paper/2106.09947/) (2021)
- [Adversarial Attack on Attackers: Post-Process to Mitigate Black-Box Score-Based Query Attacks](https://mlsanity.com/paper/2205.12134/) (2022)
- [A2: Efficient Automated Attacker for Boosting Adversarial Training](https://mlsanity.com/paper/2210.03543/) (2022)
- [Meta-Learning the Search Distribution of Black-Box Random Search Based Adversarial Attacks](https://mlsanity.com/paper/2111.01714/) (2021)
- [Improving Fast Minimum-Norm Attacks with Hyperparameter Optimization](https://mlsanity.com/paper/2310.08177/) (2023)
- [Towards Robust Vision Transformer via Masked Adaptive Ensemble](https://mlsanity.com/paper/2407.15385/) (2024)
- [Achieving Domain-Independent Certified Robustness via Knowledge Continuity](https://mlsanity.com/paper/2411.01644/) (2024)
- [Adversarial Attack and Defense for Non-Parametric Two-Sample Tests](https://mlsanity.com/paper/2202.03077/) (2022)
- [RNN-Guard: Certified Robustness Against Multi-frame Attacks for Recurrent Neural Networks](https://mlsanity.com/paper/2304.07980/) (2023)
- [On the Feasibility of Poisoning Text-to-Image AI Models via Adversarial Mislabeling](https://mlsanity.com/paper/2506.21874/) (2025)
- [Fortifying Time Series: DTW-Certified Robust Anomaly Detection](https://mlsanity.com/paper/2605.07690/) (2026)
- [A Multi-objective Memetic Algorithm for Auto Adversarial Attack Optimization Design](https://mlsanity.com/paper/2208.06984/) (2022)
- [A Minimax Approach Against Multi-Armed Adversarial Attacks Detection](https://mlsanity.com/paper/2302.02216/) (2023)
- [Multi-objective Evolutionary Search of Variable-length Composite Semantic Perturbations](https://mlsanity.com/paper/2307.06548/) (2023)
- [Redesigning Traffic Signs to Mitigate Machine-Learning Patch Attacks](https://mlsanity.com/paper/2402.04660/) (2024)
- [Reinforced Compressive Neural Architecture Search for Versatile Adversarial Robustness](https://mlsanity.com/paper/2406.06792/) (2024)
- [DataFreeShield: Defending Adversarial Attacks without Training Data](https://mlsanity.com/paper/2406.15635/) (2024)
- [HO-FMN: Hyperparameter Optimization for Fast Minimum-Norm Attacks](https://mlsanity.com/paper/2407.08806/) (2024)
- [SoK: Systematic analysis of adversarial threats against deep learning approaches for autonomous anomaly detection systems in SDN-IoT networks](https://mlsanity.com/paper/2509.26350/) (2025)
- [Dashed Line Defense: Plug-And-Play Defense Against Adaptive Score-Based Query Attacks](https://mlsanity.com/paper/2602.08679/) (2026)
- [Adversarial Frontiers: Minimum-Norm Attack Ensembles for Robustness Evaluation](https://mlsanity.com/paper/2607.19855/) (2026)

## Abstract (arXiv)

Reliable evaluation of adversarial defenses is a challenging task, currently limited to an expert who manually crafts attacks that exploit the defense's inner workings or approaches based on an ensemble of fixed attacks, none of which may be effective for the specific defense at hand. Our key observation is that adaptive attacks are composed of reusable building blocks that can be formalized in a search space and used to automatically discover attacks for unknown defenses. We evaluated our approach on 24 adversarial defenses and show that it outperforms AutoAttack, the current state-of-the-art tool for reliable evaluation of adversarial defenses: our tool discovered significantly stronger attacks by producing 3.0%-50.8% additional adversarial examples for 10 models, while obtaining attacks with slightly stronger or similar strength for the remaining models.