# Stealthy Multi-Task Adversarial Attacks

arXiv: https://arxiv.org/abs/2411.17936 · HTML: https://mlsanity.com/paper/2411.17936/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 43 references that are in the map.
- First posted on arXiv 26 Nov 2024.

## Builds on (most cited first, 24 of 24)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Multi-Task Learning as Multi-Objective Optimization](https://mlsanity.com/paper/1810.04650/) (2018)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Multi-Task Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2009.09796/) (2020)
- [Auto-Lambda: Disentangling Dynamic Task Relationships](https://mlsanity.com/paper/2202.03091/) (2022)
- [Mind the box: l1-APGD for sparse adversarial attacks on image classifiers](https://mlsanity.com/paper/2103.01208/) (2021)
- [Adversarial Attacks are Reversible with Natural Supervision](https://mlsanity.com/paper/2103.14222/) (2021)
- [A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods](https://mlsanity.com/paper/2204.03508/) (2022)
- [Physical Adversarial Attacks on an Aerial Imagery Object Detector](https://mlsanity.com/paper/2108.11765/) (2021)
- [Globally Normalized Transition-Based Neural Networks](https://mlsanity.com/paper/1603.06042/) (2016)
- [Unleashing the Power of Multi-Task Learning: A Comprehensive Survey Spanning Traditional, Deep, and Pretrained Foundation Model Eras](https://mlsanity.com/paper/2404.18961/) (2024)
- [Adversarial Robustness in Multi-Task Learning: Promises and Illusions](https://mlsanity.com/paper/2110.15053/) (2021)
- [Deep Transfer Learning for Intelligent Vehicle Perception: a Survey](https://mlsanity.com/paper/2306.15110/) (2023)
- [Defense against Adversarial Cloud Attack on Remote Sensing Salient Object Detection](https://mlsanity.com/paper/2306.17431/) (2023)
- [Multi-Task Adversarial Attack](https://mlsanity.com/paper/2011.09824/) (2020)
- [Semantically Stealthy Adversarial Attacks against Segmentation Models](https://mlsanity.com/paper/2104.01732/) (2021)
- [Adversarial Attacks on Hidden Tasks in Multi-Task Learning](https://mlsanity.com/paper/2405.15244/) (2024)
- [Attacking All Tasks at Once Using Adversarial Examples in Multi-Task Learning](https://mlsanity.com/paper/2305.12066/) (2023)

## Built on by (most cited first, 1 of 1)

- [Bridging the Task Gap: Multi-Task Adversarial Transferability in CLIP and Its Derivatives](https://mlsanity.com/paper/2509.23917/) (2025)

## Abstract (arXiv)

Deep neural networks are highly vulnerable to adversarial perturbations, raising serious safety concerns in the real-world systems. While prior work mainly explores single-task attacks or jointly degrading all tasks in multi-task models, practical scenarios often demand more selective and stealthy attack strategies. To address this challenge, we propose Stealthy Multi-Task Adversarial Attack (SMTA2), a novel framework that selectively degrades a targeted task while strictly preserving the performance of non-targeted tasks. We formulate this objective as a constrained multi-objective optimization problem and design task-aware adversarial perturbations that maximize degradation on the targeted task without causing collateral damage on non-targeted tasks. To enhance practicality, we further introduce an automated loss-weight tuning strategy that dynamically balances attack and preservation objectives. Experiments on two multi-task benchmarks NYUv2 and Cityscapes demonstrate that SMTA2 achieves strong attack performance on targeted tasks while maintaining non-targeted tasks intact on both undefended and adversarially trained models, establishing the first systematic framework for stealthy and selective multi-task attack framework.