# Robustness May Be at Odds with Fairness: An Empirical Study on Class-wise Accuracy

arXiv: https://arxiv.org/abs/2010.13365 · HTML: https://mlsanity.com/paper/2010.13365/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Oct 2020.

## Built on by (most cited first, 30 of 45)

- [CFA: Class-wise Calibrated Fair Adversarial Training](https://mlsanity.com/paper/2303.14460/) (2023)
- [Analysis and Applications of Class-wise Robustness in Adversarial Training](https://mlsanity.com/paper/2105.14240/) (2021)
- [WAT: Improve the Worst-class Robustness in Adversarial Training](https://mlsanity.com/paper/2302.04025/) (2023)
- [Beyond Performance: Quantifying and Mitigating Label Bias in LLMs](https://mlsanity.com/paper/2405.02743/) (2024)
- [Unlocking Accuracy and Fairness in Differentially Private Image Classification](https://mlsanity.com/paper/2308.10888/) (2023)
- [Universal Adversarial Training with Class-Wise Perturbations](https://mlsanity.com/paper/2104.03000/) (2021)
- [Motion-blurred Video Interpolation and Extrapolation](https://mlsanity.com/paper/2103.02984/) (2021)
- [DAFA: Distance-Aware Fair Adversarial Training](https://mlsanity.com/paper/2401.12532/) (2024)
- [Towards Fairness-Aware Adversarial Learning](https://mlsanity.com/paper/2402.17729/) (2024)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [Understanding the Impact of Adversarial Robustness on Accuracy Disparity](https://mlsanity.com/paper/2211.15762/) (2022)
- [A Comprehensive Study on Dataset Distillation: Performance, Privacy, Robustness and Fairness](https://mlsanity.com/paper/2305.03355/) (2023)
- [Improving Adversarial Robust Fairness via Anti-Bias Soft Label Distillation](https://mlsanity.com/paper/2312.05508/) (2023)
- [DRoP: Distributionally Robust Data Pruning](https://mlsanity.com/paper/2404.05579/) (2024)
- [Implementing Responsible AI: Tensions and Trade-Offs Between Ethics Aspects](https://mlsanity.com/paper/2304.08275/) (2023)
- [SoK: Unintended Interactions among Machine Learning Defenses and Risks](https://mlsanity.com/paper/2312.04542/) (2023)
- [Understanding Implosion in Text-to-Image Generative Models](https://mlsanity.com/paper/2409.12314/) (2024)
- [Trustworthy Machine Learning via Memorization and the Granular Long-Tail: A Survey on Interactions, Tradeoffs, and Beyond](https://mlsanity.com/paper/2503.07501/) (2025)
- [Security Concerns on Machine Learning Solutions for 6G Networks in mmWave Beam Prediction](https://mlsanity.com/paper/2105.03905/) (2021)
- [FAIR-TAT: Improving Model Fairness Using Targeted Adversarial Training](https://mlsanity.com/paper/2410.23142/) (2024)
- [Trustworthy AI Must Account for Interactions](https://mlsanity.com/paper/2504.07170/) (2025)
- [On the uncertainty principle of neural networks](https://mlsanity.com/paper/2205.01493/) (2022)
- [Quantum-Inspired Analysis of Neural Network Vulnerabilities: The Role of Conjugate Variables in System Attacks](https://mlsanity.com/paper/2402.10983/) (2024)
- [Improving Transformer Performance for French Clinical Notes Classification Using Mixture of Experts on a Limited Dataset](https://mlsanity.com/paper/2303.12892/) (2023)
- [Towards Class-wise Robustness Analysis](https://mlsanity.com/paper/2411.19853/) (2024)
- [FragileFlow: Spectral Control of Correct-but-Fragile Predictions for Foundation Model Robustness](https://mlsanity.com/paper/2605.08896/) (2026)
- [Holistic Approach to Measure Sample-level Adversarial Vulnerability and its Utility in Building Trustworthy Systems](https://mlsanity.com/paper/2205.02604/) (2022)
- [UnbiasedNets: A Dataset Diversification Framework for Robustness Bias Alleviation in Neural Networks](https://mlsanity.com/paper/2302.12538/) (2023)
- [Neural Dehydration: Effective Erasure of Black-box Watermarks from DNNs with Limited Data](https://mlsanity.com/paper/2309.03466/) (2023)
- [Panda or not Panda? Understanding Adversarial Attacks with Interactive Visualization](https://mlsanity.com/paper/2311.13656/) (2023)

## Abstract (arXiv)

Convolutional neural networks (CNNs) have made significant advancement, however, they are widely known to be vulnerable to adversarial attacks. Adversarial training is the most widely used technique for improving adversarial robustness to strong white-box attacks. Prior works have been evaluating and improving the model average robustness without class-wise evaluation. The average evaluation alone might provide a false sense of robustness. For example, the attacker can focus on attacking the vulnerable class, which can be dangerous, especially, when the vulnerable class is a critical one, such as "human" in autonomous driving. We propose an empirical study on the class-wise accuracy and robustness of adversarially trained models. We find that there exists inter-class discrepancy for accuracy and robustness even when the training dataset has an equal number of samples for each class. For example, in CIFAR10, "cat" is much more vulnerable than other classes. Moreover, this inter-class discrepancy also exists for normally trained models, while adversarial training tends to further increase the discrepancy. Our work aims to investigate the following questions: (a) is the phenomenon of inter-class discrepancy universal regardless of datasets, model architectures and optimization hyper-parameters? (b) If so, what can be possible explanations for the inter-class discrepancy? (c) Can the techniques proposed in the long tail classification be readily extended to adversarial training for addressing the inter-class discrepancy?