# Optimization for Robustness Evaluation beyond ℓp Metrics

arXiv: https://arxiv.org/abs/2210.00621 · HTML: https://mlsanity.com/paper/2210.00621/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 2 Oct 2022.

## Builds on (most cited first, 22 of 22)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack](https://mlsanity.com/paper/1907.02044/) (2019)
- [Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples](https://mlsanity.com/paper/2010.03593/) (2020)
- [Spatially Transformed Adversarial Examples](https://mlsanity.com/paper/1801.02612/) (2018)
- [Perceptual Adversarial Robustness: Defense Against Unseen Threat Models](https://mlsanity.com/paper/2006.12655/) (2020)
- [Sparse and Imperceivable Adversarial Attacks](https://mlsanity.com/paper/1909.05040/) (2019)
- [Wasserstein Adversarial Examples via Projected Sinkhorn Iterations](https://mlsanity.com/paper/1902.07906/) (2019)
- [Adversarial Robustness Against the Union of Multiple Perturbation Models](https://mlsanity.com/paper/1909.04068/) (2019)
- [Semantic Adversarial Examples](https://mlsanity.com/paper/1804.00499/) (2018)
- [Functional Adversarial Attacks](https://mlsanity.com/paper/1906.00001/) (2019)
- [Revisiting and Advancing Fast Adversarial Training Through The Lens of Bi-Level Optimization](https://mlsanity.com/paper/2112.12376/) (2021)
- [Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraints](https://mlsanity.com/paper/2102.12827/) (2021)
- [Logit Pairing Methods Can Fool Gradient-Based Attacks](https://mlsanity.com/paper/1810.12042/) (2018)
- [Gradient Sampling Methods for Nonsmooth Optimization](https://mlsanity.com/paper/1804.11003/) (2018)
- [GENO -- GENeric Optimization for Classical Machine Learning](https://mlsanity.com/paper/1905.13587/) (2019)
- [NCVX: A User-Friendly and Scalable Package for Nonconvex Optimization in Machine Learning](https://mlsanity.com/paper/2111.13984/) (2021)

## Built on by (most cited first, 3 of 3)

- [NCVX: A General-Purpose Optimization Solver for Constrained Machine and Deep Learning](https://mlsanity.com/paper/2210.00973/) (2022)
- [Optimization and Optimizers for Adversarial Robustness](https://mlsanity.com/paper/2303.13401/) (2023)
- [REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations](https://mlsanity.com/paper/2605.12813/) (2026)

## Abstract (arXiv)

Empirical evaluation of deep learning models against adversarial attacks entails solving nontrivial constrained optimization problems. Popular algorithms for solving these constrained problems rely on projected gradient descent (PGD) and require careful tuning of multiple hyperparameters. Moreover, PGD can only handle ℓ1, ℓ2, and ℓ_∞ attack models due to the use of analytical projectors. In this paper, we introduce a novel algorithmic framework that blends a general-purpose constrained-optimization solver PyGRANSO, With Constraint-Folding (PWCF), to add reliability and generality to robustness evaluation. PWCF 1) finds good-quality solutions without the need of delicate hyperparameter tuning, and 2) can handle general attack models, e.g., general ℓp (p ≥ 0) and perceptual attacks, which are inaccessible to PGD-based algorithms.