# TaFD: Threat-Aware Frequency Decoupling for Adversarial Robustness against Heterogeneous Attacks

arXiv: https://arxiv.org/abs/2606.17540 · HTML: https://mlsanity.com/paper/2606.17540/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 44 references that are in the map.
- First posted on arXiv 16 Jun 2026.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer](https://mlsanity.com/paper/2110.02178/) (2021)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Spatially Transformed Adversarial Examples](https://mlsanity.com/paper/1801.02612/) (2018)
- [Generating Natural Adversarial Examples](https://mlsanity.com/paper/1710.11342/) (2017)
- [Adversarial Training and Robustness for Multiple Perturbations](https://mlsanity.com/paper/1904.13000/) (2019)
- [Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art](https://mlsanity.com/paper/1704.05519/) (2017)
- [Perceptual Adversarial Robustness: Defense Against Unseen Threat Models](https://mlsanity.com/paper/2006.12655/) (2020)
- [SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing](https://mlsanity.com/paper/1906.07927/) (2019)
- [ColorFool: Semantic Adversarial Colorization](https://mlsanity.com/paper/1911.10891/) (2019)
- [Semantic Adversarial Examples](https://mlsanity.com/paper/1804.00499/) (2018)
- [Unrestricted Adversarial Examples via Semantic Manipulation](https://mlsanity.com/paper/1904.06347/) (2019)
- [Functional Adversarial Attacks](https://mlsanity.com/paper/1906.00001/) (2019)
- [Adversarial Robustness against Multiple and Single lp-Threat Models via Quick Fine-Tuning of Robust Classifiers](https://mlsanity.com/paper/2105.12508/) (2021)
- [Adversarial Defense by Suppressing High-frequency Components](https://mlsanity.com/paper/1908.06566/) (2019)
- [Robustness and Generalization via Generative Adversarial Training](https://mlsanity.com/paper/2109.02765/) (2021)
- [Understanding Adversarial Robustness Against On-manifold Adversarial Examples](https://mlsanity.com/paper/2210.00430/) (2022)
- [DAT: Improving Adversarial Robustness via Generative Amplitude Mix-up in Frequency Domain](https://mlsanity.com/paper/2410.12307/) (2024)
- [ALA: Naturalness-aware Adversarial Lightness Attack](https://mlsanity.com/paper/2201.06070/) (2022)
- [Provable Unrestricted Adversarial Training without Compromise with Generalizability](https://mlsanity.com/paper/2301.09069/) (2023)

## Abstract (arXiv)

Multi-threat robustness remains a fundamental challenge in deep learning. Although joint adversarial training (JAT) is widely adopted, it suffers from negative transfer under heterogeneous threats, particularly between ℓp-bounded and semantic attacks. Through first-order gradient analysis, we formalize this as gradient incompatibility and theoretically establish the necessity of decoupled optimization. We further reveal that these conflicting threats exhibit separable spectral characteristics in the frequency domain. Motivated by this observation, we propose Threat-aware Frequency Decoupling (TaFD), a two-stage defense framework that reformulates JAT as a frequency-domain divide-and-conquer paradigm. TaFD first discovers latent threat domains via unsupervised clustering of attack spectral prototypes and trains a lightweight classifier for inference-time threat domain identification. Conditioned on the prediction, TaFD employs a Frequency-Conditional Convolution that learns threat-domain-specific spectral masks and routes each sample to the corresponding expert, enforcing structural parameter separation and alleviating optimization conflicts. We validate TaFD on three representative image-classification benchmarks (CIFAR-10, CIFAR-100, and Tiny-ImageNet) and on two representative architectures (the convolutional ResNet and the hybrid-transformer MobileViT). Extensive results demonstrate that TaFD achieves more balanced robustness against heterogeneous attacks than existing JAT and frequency-domain baselines, improving average robust accuracy by approximately 11% over the strongest baseline while maintaining leading clean accuracy.