# Adversarial Attacks on Hidden Tasks in Multi-Task Learning

arXiv: https://arxiv.org/abs/2405.15244 · HTML: https://mlsanity.com/paper/2405.15244/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 40 references that are in the map.
- First posted on arXiv 24 May 2024.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [Feature Importance-aware Transferable Adversarial Attacks](https://mlsanity.com/paper/2107.14185/) (2021)
- [HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition](https://mlsanity.com/paper/1603.01249/) (2016)
- [Enhancing Cross-task Black-Box Transferability of Adversarial Examples with Dispersion Reduction](https://mlsanity.com/paper/1911.11616/) (2019)
- [The iMaterialist Fashion Attribute Dataset](https://mlsanity.com/paper/1906.05750/) (2019)
- [Adversarial Example Games](https://mlsanity.com/paper/2007.00720/) (2020)
- [Practical No-box Adversarial Attacks against DNNs](https://mlsanity.com/paper/2012.02525/) (2020)
- [Task-generalizable Adversarial Attack based on Perceptual Metric](https://mlsanity.com/paper/1811.09020/) (2018)
- [Attacking deep networks with surrogate-based adversarial black-box methods is easy](https://mlsanity.com/paper/2203.08725/) (2022)
- [Adversarial Robustness in Multi-Task Learning: Promises and Illusions](https://mlsanity.com/paper/2110.15053/) (2021)
- [Adversarial Attacks on Multi-task Visual Perception for Autonomous Driving](https://mlsanity.com/paper/2107.07449/) (2021)
- [Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient](https://mlsanity.com/paper/2308.05681/) (2023)
- [Practical No-box Adversarial Attacks with Training-free Hybrid Image Transformation](https://mlsanity.com/paper/2203.04607/) (2022)
- [Multi-Task Adversarial Attack](https://mlsanity.com/paper/2011.09824/) (2020)
- [UniNet: A Unified Scene Understanding Network and Exploring Multi-Task Relationships through the Lens of Adversarial Attacks](https://mlsanity.com/paper/2108.04584/) (2021)

## Built on by (most cited first, 2 of 2)

- [Stealthy Multi-Task Adversarial Attacks](https://mlsanity.com/paper/2411.17936/) (2024)
- [One Object, Multiple Lies: A Benchmark for Cross-task Adversarial Attack on Unified Vision-Language Models](https://mlsanity.com/paper/2507.07709/) (2025)

## Abstract (arXiv)

Deep learning models are susceptible to adversarial attacks, where slight perturbations to input data lead to misclassification. Adversarial attacks become increasingly effective with access to information about the targeted classifier. In the context of multi-task learning, where a single model learns multiple tasks simultaneously, attackers may aim to exploit vulnerabilities in specific tasks with limited information. This paper investigates the feasibility of attacking hidden tasks within multi-task classifiers, where model access regarding the hidden target task and labeled data for the hidden target task are not available, but model access regarding the non-target tasks is available. We propose a novel adversarial attack method that leverages knowledge from non-target tasks and the shared backbone network of the multi-task model to force the model to forget knowledge related to the target task. Experimental results on CelebA and DeepFashion datasets demonstrate the effectiveness of our method in degrading the accuracy of hidden tasks while preserving the performance of visible tasks, contributing to the understanding of adversarial vulnerabilities in multi-task classifiers.