mlsanityData to 14 Oct 2026

Towards Class-wise Fair Adversarial Training via Anti-Bias Soft Label Distillation

arXiv 2506.08611 ↗First posted 10 Jun 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Adversarial Training (AT) is widely recognized as an effective approach to enhance the adversarial robustness of Deep Neural Networks. As a variant of AT, Adversarial Robustness Distillation (ARD) has shown outstanding performance in enhancing the robustness of small models. However, both AT and ARD face robust fairness issue: these models tend to display strong adversarial robustness against some classes (easy classes) while demonstrating weak adversarial robustness against others (hard classes). This paper explores the underlying factors of this problem and points out the smoothness degree of soft labels for different classes significantly impacts the robust fairness from both empirical observation and theoretical analysis. Based on the above exploration, we propose Anti-Bias Soft Label Distillation (ABSLD) within the Knowledge Distillation framework to enhance the adversarial robust fairness. Specifically, ABSLD adaptively reduces the student's error risk gap between different classes, which is accomplished by adjusting the class-wise smoothness degree of teacher's soft labels during the training process, and the adjustment is managed by assigning varying temperatures to different classes. Additionally, as a label-based approach, ABSLD is highly adaptable and can be integrated with the sample-based methods. Extensive experiments demonstrate ABSLD outperforms state-of-the-art methods on the comprehensive performance of robustness and fairness.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 49 references that are in the map. First posted on arXiv 10 Jun 2025.

Builds on

38 of 49 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  6. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  7. 2013Intriguing properties of neural networkscited by 5,155
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  9. 2016Wide Residual Networkscited by 3,070
  10. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  11. 2015Fast R-CNNcited by 2,933
  12. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
26 more
  1. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  2. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  3. 2020Overfitting in adversarially robust deep learningcited by 506
  4. 2019Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attackcited by 334
  5. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 278
  6. 2021Data Augmentation Can Improve Robustnesscited by 194
  7. 2020To be Robust or to be Fair: Towards Fairness in Adversarial Trainingcited by 137
  8. 2019Adversarially Robust Distillationcited by 125
  9. 2021Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?cited by 113
  10. 2020Boosting Adversarial Training with Hypersphere Embeddingcited by 97
  11. 2022LAS-AT: Adversarial Training with Learnable Attack Strategycited by 95
  12. 2021Adversarial Sticker: A Stealthy Attack Method in the Physical Worldcited by 93
  13. 2021Reliable Adversarial Distillation with Unreliable Teacherscited by 56
  14. 2023CFA: Class-wise Calibrated Fair Adversarial Trainingcited by 53
  15. 2020Robustness May Be at Odds with Fairness: An Empirical Study on Class-wise Accuracycited by 45
  16. 2022Simultaneously Optimizing Perturbations and Positions for Black-box Adversarial Patch Attackscited by 42
  17. 2022Prior-Guided Adversarial Initialization for Fast Adversarial Trainingcited by 35
  18. 2023Improving Fast Adversarial Training with Prior-Guided Knowledgecited by 35
  19. 2021Analysis and Applications of Class-wise Robustness in Adversarial Trainingcited by 33
  20. 2023WAT: Improve the Worst-class Robustness in Adversarial Trainingcited by 27
  21. 2023Efficient Robustness Assessment via Adversarial Spatial-Temporal Focus on Videoscited by 20
  22. 2023Mitigating Accuracy-Robustness Trade-off via Balanced Multi-Teacher Adversarial Distillationcited by 13
  23. 2024DAFA: Distance-Aware Fair Adversarial Trainingcited by 11
  24. 2023Improving Adversarial Robust Fairness via Anti-Bias Soft Label Distillationcited by 8
  25. 2023Improving Viewpoint Robustness for Visual Recognition via Adversarial Trainingcited by 7
  26. 2023Revisiting the Trade-off between Accuracy and Robustness via Weight Distribution of Filterscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better
  2. 2022On the benefits of knowledge distillation for adversarial robustness
  3. 2022ARDIR: Improving Robustness using Knowledge Distillation of Internal Representation
  4. 2023Enhancing Adversarial Robustness in Low-Label Regime via Adaptively Weighted Regularization and Knowledge Distillation
  5. 2024Distilling Adversarial Robustness Using Heterogeneous Teachers
  6. 2026Toward Understanding Adversarial Distillation: Why Robust Teachers Fail
  7. 2026Learning Better Certified Models from Empirically-Robust Teachers
  8. 2022Vanilla Feature Distillation for Improving the Accuracy-Robustness Trade-Off in Adversarial Training