# On the Hardness of Robustness Transfer: A Perspective from Rademacher Complexity over Symmetric Difference Hypothesis Space

arXiv: https://arxiv.org/abs/2302.12351 · HTML: https://mlsanity.com/paper/2302.12351/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 40 references that are in the map.
- First posted on arXiv 23 Feb 2023.

## Builds on (most cited first, 20 of 20)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Maximum Classifier Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1712.02560/) (2017)
- [An Introduction to Matrix Concentration Inequalities](https://mlsanity.com/paper/1501.01571/) (2015)
- [Bridging Theory and Algorithm for Domain Adaptation](https://mlsanity.com/paper/1904.05801/) (2019)
- [Do Adversarially Robust ImageNet Models Transfer Better?](https://mlsanity.com/paper/2007.08489/) (2020)
- [Rademacher Complexity for Adversarially Robust Generalization](https://mlsanity.com/paper/1810.11914/) (2018)
- [VC Classes are Adversarially Robustly Learnable, but Only Improperly](https://mlsanity.com/paper/1902.04217/) (2019)
- [When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?](https://mlsanity.com/paper/2111.01124/) (2021)
- [Adversarially robust transfer learning](https://mlsanity.com/paper/1905.08232/) (2019)
- [Adversarial Learning Guarantees for Linear Hypotheses and Neural Networks](https://mlsanity.com/paper/2004.13617/) (2020)
- [Adversarial Training Helps Transfer Learning via Better Representations](https://mlsanity.com/paper/2106.10189/) (2021)
- [Adversarial Risk and Robustness: General Definitions and Implications for the Uniform Distribution](https://mlsanity.com/paper/1810.12272/) (2018)
- [The Many Faces of Adversarial Risk](https://mlsanity.com/paper/2201.08956/) (2022)
- [On the Hardness of Robust Classification](https://mlsanity.com/paper/1909.05822/) (2019)
- [Unsupervised Domain Adaptation Based on Source-guided Discrepancy](https://mlsanity.com/paper/1809.03839/) (2018)
- [On Localized Discrepancy for Domain Adaptation](https://mlsanity.com/paper/2008.06242/) (2020)
- [Lower Bounds for Adversarially Robust PAC Learning](https://mlsanity.com/paper/1906.05815/) (2019)
- [Adaptation Algorithm and Theory Based on Generalized Discrepancy](https://mlsanity.com/paper/1405.1503/) (2014)

## Built on by (most cited first, 2 of 2)

- [Reliable learning in challenging environments](https://mlsanity.com/paper/2304.03370/) (2023)
- [Leveraging Pre-trained Models for Robust Federated Learning for Kidney Stone Type Recognition](https://mlsanity.com/paper/2409.19934/) (2024)

## Abstract (arXiv)

Recent studies demonstrated that the adversarially robust learning under ℓ_∞ attack is harder to generalize to different domains than standard domain adaptation. How to transfer robustness across different domains has been a key question in domain adaptation field. To investigate the fundamental difficulty behind adversarially robust domain adaptation (or robustness transfer), we propose to analyze a key complexity measure that controls the cross-domain generalization: the adversarial Rademacher complexity over em symmetric difference hypothesis space H Delta H. For linear models, we show that adversarial version of this complexity is always greater than the non-adversarial one, which reveals the intrinsic hardness of adversarially robust domain adaptation. We also establish upper bounds on this complexity measure. Then we extend them to the ReLU neural network class by upper bounding the adversarial Rademacher complexity in the binary classification setting. Finally, even though the robust domain adaptation is provably harder, we do find positive relation between robust learning and standard domain adaptation. We explain how adversarial training helps domain adaptation in terms of standard risk. We believe our results initiate the study of the generalization theory of adversarially robust domain adaptation, and could shed lights on distributed adversarially robust learning from heterogeneous sources, e.g., federated learning scenario.