# Adversarial Robustness in Multi-Task Learning: Promises and Illusions

arXiv: https://arxiv.org/abs/2110.15053 · HTML: https://mlsanity.com/paper/2110.15053/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 26 Oct 2021.

## Builds on (most cited first, 21 of 21)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [Which Tasks Should Be Learned Together in Multi-task Learning?](https://mlsanity.com/paper/1905.07553/) (2019)
- [Certifying Some Distributional Robustness with Principled Adversarial Training](https://mlsanity.com/paper/1710.10571/) (2017)
- [On Detecting Adversarial Perturbations](https://mlsanity.com/paper/1702.04267/) (2017)
- [The Space of Transferable Adversarial Examples](https://mlsanity.com/paper/1704.03453/) (2017)
- [On the Robustness of Semantic Segmentation Models to Adversarial Attacks](https://mlsanity.com/paper/1711.09856/) (2017)
- [Branched Multi-Task Networks: Deciding What Layers To Share](https://mlsanity.com/paper/1904.02920/) (2019)
- [VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry](https://mlsanity.com/paper/1804.08366/) (2018)
- [Adversarial Embedding: A robust and elusive Steganography and Watermarking technique](https://mlsanity.com/paper/1912.01487/) (2019)

## Built on by (most cited first, 17 of 17)

- [Sibling-Attack: Rethinking Transferable Adversarial Attacks against Face Recognition](https://mlsanity.com/paper/2303.12512/) (2023)
- [A Unified Framework for Adversarial Attack and Defense in Constrained Feature Space](https://mlsanity.com/paper/2112.01156/) (2021)
- [Detecting Adversarial Perturbations in Multi-Task Perception](https://mlsanity.com/paper/2203.01177/) (2022)
- [SafePowerGraph: Safety-aware Evaluation of Graph Neural Networks for Transmission Power Grids](https://mlsanity.com/paper/2407.12421/) (2024)
- [GAT: Guided Adversarial Training with Pareto-optimal Auxiliary Tasks](https://mlsanity.com/paper/2302.02907/) (2023)
- [On Evaluating Adversarial Robustness of Chest X-ray Classification: Pitfalls and Best Practices](https://mlsanity.com/paper/2212.08130/) (2022)
- [A Multi-task Adversarial Attack Against Face Authentication](https://mlsanity.com/paper/2408.08205/) (2024)
- [SkeleVision: Towards Adversarial Resiliency of Person Tracking with Multi-Task Learning](https://mlsanity.com/paper/2204.00734/) (2022)
- [Adversarial Attacks on Hidden Tasks in Multi-Task Learning](https://mlsanity.com/paper/2405.15244/) (2024)
- [Merge Now, Regret Later: The Hidden Cost of Model Merging Is Adversarial Transferability](https://mlsanity.com/paper/2509.23689/) (2025)
- [Multitasking Models are Robust to Structural Failure: A Neural Model for Bilingual Cognitive Reserve](https://mlsanity.com/paper/2210.11618/) (2022)
- [Attacking All Tasks at Once Using Adversarial Examples in Multi-Task Learning](https://mlsanity.com/paper/2305.12066/) (2023)
- [Stealthy Multi-Task Adversarial Attacks](https://mlsanity.com/paper/2411.17936/) (2024)
- [Multi-task Adversarial Attacks against Black-box Model with Few-shot Queries](https://mlsanity.com/paper/2508.10039/) (2025)
- [RobuMTL: Enhancing Multi-Task Learning Robustness Against Weather Conditions](https://mlsanity.com/paper/2601.10921/) (2026)
- [Hear No Evil: Towards Adversarial Robustness of Automatic Speech Recognition via Multi-Task Learning](https://mlsanity.com/paper/2204.02381/) (2022)
- [Multi-Task Consistency-based Detection of Adversarial Attacks](https://mlsanity.com/paper/2608.07750/) (2026)

## Abstract (arXiv)

Vulnerability to adversarial attacks is a well-known weakness of Deep Neural networks. While most of the studies focus on single-task neural networks with computer vision datasets, very little research has considered complex multi-task models that are common in real applications. In this paper, we evaluate the design choices that impact the robustness of multi-task deep learning networks. We provide evidence that blindly adding auxiliary tasks, or weighing the tasks provides a false sense of robustness. Thereby, we tone down the claim made by previous research and study the different factors which may affect robustness. In particular, we show that the choice of the task to incorporate in the loss function are important factors that can be leveraged to yield more robust models.