mlsanityData to 14 Oct 2026

FERD: Fairness-Enhanced Data-Free Robustness Distillation

arXiv 2509.20793 ↗First posted 25 Sep 2025cs.LG cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Data-Free Robustness Distillation (DFRD) aims to transfer the robustness from the teacher to the student without accessing the training data. While existing methods focus on overall robustness, they overlook the robust fairness issues, leading to severe disparity of robustness across different categories. In this paper, we find two key problems: (1) student model distilled with equal class proportion data behaves significantly different across distinct categories; and (2) the robustness of student model is not stable across different attacks target. To bridge these gaps, we present the first Fairness-Enhanced data-free Robustness Distillation (FERD) framework to adjust the proportion and distribution of adversarial examples. For the proportion, FERD adopts a robustness-guided class reweighting strategy to synthesize more samples for the less robust categories, thereby improving robustness of them. For the distribution, FERD generates complementary data samples for advanced robustness distillation. It generates Fairness-Aware Examples (FAEs) by enforcing a uniformity constraint on feature-level predictions, which suppress the dominance of class-specific non-robust features, providing a more balanced representation across all categories. Then, FERD constructs Uniform-Target Adversarial Examples (UTAEs) from FAEs by applying a uniform target class constraint to avoid biased attack directions, which distribute the attack targets across all categories and prevents overfitting to specific vulnerable categories. Extensive experiments on three public datasets show that FERD achieves state-of-the-art worst-class robustness under all adversarial attack (e.g., the worst-class robustness under FGSM and AutoAttack are improved by 15.1% and 6.4% using MobileNet-V2 on CIFAR-10), demonstrating superior performance in both robustness and fairness aspects.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 49 references that are in the map. First posted on arXiv 25 Sep 2025.

Builds on

36 of 49 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  5. 2016Wide Residual Networkscited by 3,070
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  7. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  8. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  9. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 423
  10. 2019Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systemscited by 153
  11. 2020To be Robust or to be Fair: Towards Fairness in Adversarial Trainingcited by 137
  12. 2019Zero-shot Knowledge Transfer via Adversarial Belief Matchingcited by 129
24 more
  1. 2019Adversarially Robust Distillationcited by 125
  2. 2019Adversarial Robustness vs Model Compression, or Both?cited by 98
  3. 2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 95
  4. 2021Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Bettercited by 79
  5. 2022Parallel Rectangle Flip Attack: A Query-based Black-box Attack against Object Detectioncited by 79
  6. 2021Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networkscited by 74
  7. 2020Efficient Adversarial Attacks for Visual Object Trackingcited by 66
  8. 2021Contrastive Model Inversion for Data-Free Knowledge Distillationcited by 66
  9. 2023X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detectioncited by 59
  10. 2021Reliable Adversarial Distillation with Unreliable Teacherscited by 56
  11. 2022A Large-scale Multiple-objective Method for Black-box Attack against Object Detectioncited by 55
  12. 2024Lightweight Deep Learning for Resource-Constrained Environments: A Surveycited by 38
  13. 2023Improving Fast Adversarial Training with Prior-Guided Knowledgecited by 35
  14. 2018Unravelling Robustness of Deep Learning based Face Recognition Against Adversarial Attackscited by 31
  15. 2022Distilling Robust and Non-Robust Features in Adversarial Examples by Information Bottleneckcited by 30
  16. 2021A Review of Adversarial Attack and Defense for Classification Methodscited by 29
  17. 2022Towards Compositional Adversarial Robustness: Generalizing Adversarial Training to Composite Semantic Perturbationscited by 26
  18. 2022Exploring the Relationship between Architecture and Adversarially Robust Generalizationcited by 23
  19. 2023Improving Generalization of Adversarial Training via Robust Critical Fine-Tuningcited by 23
  20. 2025E2AT: Multimodal Jailbreak Defense via Dynamic Joint Optimization for Multimodal Large Language Modelscited by 12
  21. 2023Fast Propagation is Better: Accelerating Single-Step Adversarial Training via Sampling Subnetworkscited by 8
  22. 2023Improving Adversarial Robust Fairness via Anti-Bias Soft Label Distillationcited by 8
  23. 2023Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillationcited by 7
  24. 2023Data-Free Hard-Label Robustness Stealing Attackcited by 6

Built on by

Not cited yet.