# Can Attack Difficulty Be Characterized Before Optimization? A Study of Pre-optimization Difficulty in Person-Vanishing Attacks

arXiv: https://arxiv.org/abs/2609.34501 · HTML: https://mlsanity.com/paper/2609.34501/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 26 references that are in the map.
- First posted on arXiv 28 Sep 2026.

## Builds on (most cited first, 20 of 20)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Adversarial Examples for Semantic Segmentation and Object Detection](https://mlsanity.com/paper/1703.08603/) (2017)
- [Fooling automated surveillance cameras: adversarial patches to attack person detection](https://mlsanity.com/paper/1904.08653/) (2019)
- [Attacks Which Do Not Kill Training Make Adversarial Learning Stronger](https://mlsanity.com/paper/2002.11242/) (2020)
- [Geometry-aware Instance-reweighted Adversarial Training](https://mlsanity.com/paper/2010.01736/) (2020)
- [Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach](https://mlsanity.com/paper/1801.10578/) (2018)
- [MMA Training: Direct Input Space Margin Maximization through Adversarial Training](https://mlsanity.com/paper/1812.02637/) (2018)
- [Making an Invisibility Cloak: Real World Adversarial Attacks on Object Detectors](https://mlsanity.com/paper/1910.14667/) (2019)
- [Adversarial Texture for Fooling Person Detectors in the Physical World](https://mlsanity.com/paper/2203.03373/) (2022)
- [Robustness of classifiers: from adversarial to random noise](https://mlsanity.com/paper/1608.08967/) (2016)
- [DAP: A Dynamic Adversarial Patch for Evading Person Detectors](https://mlsanity.com/paper/2305.11618/) (2023)
- [Evaluating Adversarial Attacks on Driving Safety in Vision-Based Autonomous Vehicles](https://mlsanity.com/paper/2108.02940/) (2021)
- [When Pedestrian Detection Meets Multi-Modal Learning: Generalist Model and Benchmark Dataset](https://mlsanity.com/paper/2407.10125/) (2024)
- [Selection of Source Images Heavily Influences the Effectiveness of Adversarial Attacks](https://mlsanity.com/paper/2106.07141/) (2021)
- [On Extensions of CLEVER: A Neural Network Robustness Evaluation Algorithm](https://mlsanity.com/paper/1810.08640/) (2018)
- [Adversarial Attacks on Event-Based Pedestrian Detectors: A Physical Approach](https://mlsanity.com/paper/2503.00377/) (2025)
- [Adversarial Attention Perturbations for Large Object Detection Transformers](https://mlsanity.com/paper/2508.02987/) (2025)
- [A Safety-Adapted Loss for Pedestrian Detection in Automated Driving](https://mlsanity.com/paper/2402.02986/) (2024)

## Abstract (arXiv)

Adversarial attacks against object detectors are traditionally studied from an optimization perspective, where attack difficulty is regarded as an outcome observed only after adversarial optimization. This raises a fundamental question: can the relative attack difficulty of different inputs be characterized before optimization begins? In this paper, we investigate this question for person-vanishing attacks by introducing the concept of pre-optimization attack difficulty, which captures intrinsic differences in optimization effort across input images. To estimate this latent difficulty before optimization, we propose Quad-CLEVER, an efficient geometry-based estimator derived from a quadratic approximation of the local person-vanishing margin along the most attack-relevant direction. Extensive experiments across multiple attack algorithms demonstrate that Quad-CLEVER consistently correlates with the observed optimization cost, providing empirical evidence that attack difficulty exhibits a predictable pre-optimization structure. Building upon this finding, we further propose a difficulty-aware attack framework that leverages the estimated difficulty to adaptively allocate optimization budgets for a base attack under a fixed computational budget. On BDD100K, the proposed framework improves the image-level attack success rate by up to 5.78% while reducing the average optimization cost by up to 11.42 iterations. On the more challenging EventPed dataset, it saves 2.25 optimization iterations while maintaining comparable attack performance. These results demonstrate that attack difficulty can be meaningfully estimated before optimization and that exploiting such estimates enables more computationally efficient adversarial attacks.