# Adversarially Robust Models may not Transfer Better: Sufficient Conditions for Domain Transferability from the View of Regularization

arXiv: https://arxiv.org/abs/2202.01832 · HTML: https://mlsanity.com/paper/2202.01832/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 56 references that are in the map.
- First posted on arXiv 3 Feb 2022.

## Builds on (most cited first, 30 of 33)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Optimal Transport for Domain Adaptation](https://mlsanity.com/paper/1507.00504/) (2015)
- [The Effectiveness of Data Augmentation in Image Classification using Deep Learning](https://mlsanity.com/paper/1712.04621/) (2017)
- [HopSkipJumpAttack: A Query-Efficient Decision-Based Attack](https://mlsanity.com/paper/1904.02144/) (2019)
- [Do Adversarially Robust ImageNet Models Transfer Better?](https://mlsanity.com/paper/2007.08489/) (2020)
- [Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality](https://mlsanity.com/paper/1801.02613/) (2018)
- [How Does Mixup Help With Robustness and Generalization?](https://mlsanity.com/paper/2010.04819/) (2020)
- [AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks](https://mlsanity.com/paper/1805.11770/) (2018)
- [Robust Learning with Jacobian Regularization](https://mlsanity.com/paper/1908.02729/) (2019)
- [Dataset Augmentation in Feature Space](https://mlsanity.com/paper/1702.05538/) (2017)
- [A Kernel Theory of Modern Data Augmentation](https://mlsanity.com/paper/1803.06084/) (2018)
- [On Mixup Regularization](https://mlsanity.com/paper/2006.06049/) (2020)
- [Understanding data augmentation for classification: when to warp?](https://mlsanity.com/paper/1609.08764/) (2016)
- [Data augmentation instead of explicit regularization](https://mlsanity.com/paper/1806.03852/) (2018)
- [Learning Invariances using the Marginal Likelihood](https://mlsanity.com/paper/1808.05563/) (2018)
- [TSS: Transformation-Specific Smoothing for Robustness Certification](https://mlsanity.com/paper/2002.12398/) (2020)
- [f-Domain-Adversarial Learning: Theory and Algorithms](https://mlsanity.com/paper/2106.11344/) (2021)
- [Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation](https://mlsanity.com/paper/1810.05162/) (2018)
- [TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothness](https://mlsanity.com/paper/2104.00671/) (2021)
- [Adversarial Training Helps Transfer Learning via Better Representations](https://mlsanity.com/paper/2106.10189/) (2021)
- [Controllable Orthogonalization in Training DNNs](https://mlsanity.com/paper/2004.00917/) (2020)
- [An Online Learning Approach to Interpolation and Extrapolation in Domain Generalization](https://mlsanity.com/paper/2102.13128/) (2021)

## Built on by (most cited first, 8 of 8)

- [A Survey on Transferability of Adversarial Examples across Deep Neural Networks](https://mlsanity.com/paper/2310.17626/) (2023)
- [Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferability](https://mlsanity.com/paper/2307.07873/) (2023)
- [Efficient Adversarial Contrastive Learning via Robustness-Aware Coreset Selection](https://mlsanity.com/paper/2302.03857/) (2023)
- [Reliable Evaluation of Adversarial Transferability](https://mlsanity.com/paper/2306.08565/) (2023)
- [As Firm As Their Foundations: Can open-sourced foundation models be used to create adversarial examples for downstream tasks?](https://mlsanity.com/paper/2403.12693/) (2024)
- [Improving Generalization of Universal Adversarial Perturbation via Dynamic Maximin Optimization](https://mlsanity.com/paper/2503.12793/) (2025)
- [FedGTST: Boosting Global Transferability of Federated Models via Statistics Tuning](https://mlsanity.com/paper/2410.13045/) (2024)
- [Meta-reinforcement learning with minimum attention](https://mlsanity.com/paper/2505.16741/) (2025)

## Abstract (arXiv)

Machine learning (ML) robustness and domain generalization are fundamentally correlated: they essentially concern data distribution shifts under adversarial and natural settings, respectively. On one hand, recent studies show that more robust (adversarially trained) models are more generalizable. On the other hand, there is a lack of theoretical understanding of their fundamental connections. In this paper, we explore the relationship between regularization and domain transferability considering different factors such as norm regularization and data augmentations (DA). We propose a general theoretical framework proving that factors involving the model function class regularization are sufficient conditions for relative domain transferability. Our analysis implies that ``robustness" is neither necessary nor sufficient for transferability; rather, regularization is a more fundamental perspective for understanding domain transferability. We then discuss popular DA protocols (including adversarial training) and show when they can be viewed as the function class regularization under certain conditions and therefore improve generalization. We conduct extensive experiments to verify our theoretical findings and show several counterexamples where robustness and generalization are negatively correlated on different datasets.