# Don't Waste the Noise: Importance-Guided Perturbation Allocation under Joint Global and Local Constraints

arXiv: https://arxiv.org/abs/2610.00861 · HTML: https://mlsanity.com/paper/2610.00861/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 1 Oct 2026.

## Builds on (most cited first, 20 of 20)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [RobustBench: a standardized adversarial robustness benchmark](https://mlsanity.com/paper/2010.09670/) (2020)
- [Better Diffusion Models Further Improve Adversarial Training](https://mlsanity.com/paper/2302.04638/) (2023)
- [EAD: Elastic-Net Attacks to Deep Neural Networks via Adversarial Examples](https://mlsanity.com/paper/1709.04114/) (2017)
- [Sparse and Imperceivable Adversarial Attacks](https://mlsanity.com/paper/1909.05040/) (2019)
- [SparseFool: a few pixels make a big difference](https://mlsanity.com/paper/1811.02248/) (2018)
- [Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Models](https://mlsanity.com/paper/2303.01870/) (2023)
- [Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attacks](https://mlsanity.com/paper/2006.12834/) (2020)
- [Structured Adversarial Attack: Towards General Implementation and Better Interpretability](https://mlsanity.com/paper/1808.01664/) (2018)
- [A Light Recipe to Train Robust Vision Transformers](https://mlsanity.com/paper/2209.07399/) (2022)
- [Mind the box: l1-APGD for sparse adversarial attacks on image classifiers](https://mlsanity.com/paper/2103.01208/) (2021)
- [Fast Minimum-norm Adversarial Attacks through Adaptive Norm Constraints](https://mlsanity.com/paper/2102.12827/) (2021)
- [GreedyFool: Distortion-Aware Sparse Adversarial Attack](https://mlsanity.com/paper/2010.13773/) (2020)
- [AttackBench: Evaluating Gradient-based Attacks for Adversarial Examples](https://mlsanity.com/paper/2404.19460/) (2024)
- [Saliency Attack: Towards Imperceptible Black-box Adversarial Attack](https://mlsanity.com/paper/2206.01898/) (2022)
- [MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustness](https://mlsanity.com/paper/2312.04960/) (2023)

## Abstract (arXiv)

Adversarial optimization under a shared ℓ1 budget requires deciding not only how much perturbation to use, but also where that limited budget should be spent. This allocation problem becomes particularly important when individual input coordinates are subject to local magnitude constraints, which restrict the extent to which perturbation can be concentrated on a small number of locations. We introduce an importance-guided allocation mechanism that uses a fixed clean-gradient prior to steer perturbation toward model-sensitive regions while leaving the feasible perturbation set unchanged. A centered allocation objective encourages perturbation at above-average importance locations and discourages unnecessary expenditure elsewhere, thereby redistributing rather than enlarging the available budget. Across ten robust model--dataset configurations under a common capacity-limited threat setting, the proposed method improves attack success over matched APGD- and PMA-based baselines by 2.52 to 17.70 percentage points. Allocation analysis shows that these gains are accompanied by substantially greater perturbation mass in high-importance regions without increased global ℓ1 consumption. Mechanism ablations further show that centered non-uniform redistribution provides part of the benefit, while model-derived importance yields an additional improvement. These results identify perturbation allocation as a distinct and practically relevant dimension of adversarial optimization under shared-budget, locally constrained threat models.