# Automatic Fairness Testing of Neural Classifiers through Adversarial Sampling

arXiv: https://arxiv.org/abs/2107.08176 · HTML: https://mlsanity.com/paper/2107.08176/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 64 references that are in the map.
- First posted on arXiv 17 Jul 2021.

## Builds on (most cited first, 30 of 31)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [TextBugger: Generating Adversarial Text Against Real-world Applications](https://mlsanity.com/paper/1812.05271/) (2018)
- [On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models](https://mlsanity.com/paper/1810.12715/) (2018)
- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars](https://mlsanity.com/paper/1708.08559/) (2017)
- [Ex Machina: Personal Attacks Seen at Scale](https://mlsanity.com/paper/1610.08914/) (2016)
- [Certified Robustness to Adversarial Word Substitutions](https://mlsanity.com/paper/1909.00986/) (2019)
- [Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations](https://mlsanity.com/paper/1707.00075/) (2017)
- [Counterfactual Fairness in Text Classification through Robustness](https://mlsanity.com/paper/1809.10610/) (2018)
- [Why Is My Classifier Discriminatory?](https://mlsanity.com/paper/1805.12002/) (2018)
- [Privacy Risks of Securing Machine Learning Models against Adversarial Examples](https://mlsanity.com/paper/1905.10291/) (2019)
- [Fairness Testing: Testing Software for Discrimination](https://mlsanity.com/paper/1709.03221/) (2017)
- [Guiding Deep Learning System Testing using Surprise Adequacy](https://mlsanity.com/paper/1808.08444/) (2018)
- [Crafting Adversarial Input Sequences for Recurrent Neural Networks](https://mlsanity.com/paper/1604.08275/) (2016)
- [Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation](https://mlsanity.com/paper/1909.01492/) (2019)
- [Automated Directed Fairness Testing](https://mlsanity.com/paper/1807.00468/) (2018)
- [Satisfying Real-world Goals with Dataset Constraints](https://mlsanity.com/paper/1606.07558/) (2016)
- [Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing](https://mlsanity.com/paper/1812.05793/) (2018)
- [Probabilistic Verification of Fairness Properties via Concentration](https://mlsanity.com/paper/1812.02573/) (2018)
- [RobOT: Robustness-Oriented Testing for Deep Learning Systems](https://mlsanity.com/paper/2102.05913/) (2021)
- [Robustness to Programmable String Transformations via Augmented Abstract Training](https://mlsanity.com/paper/2002.09579/) (2020)

## Built on by (most cited first, 14 of 14)

- [TESTSGD: Interpretable Testing of Neural Networks Against Subtle Group Discrimination](https://mlsanity.com/paper/2208.11321/) (2022)
- [Fairness Testing of Deep Image Classification with Adequacy Metrics](https://mlsanity.com/paper/2111.08856/) (2021)
- [Adversarial Sampling for Fairness Testing in Deep Neural Network](https://mlsanity.com/paper/2303.02874/) (2023)
- [FairRec: Fairness Testing for Deep Recommender Systems](https://mlsanity.com/paper/2304.07030/) (2023)
- [RAI4IoE: Responsible AI for Enabling the Internet of Energy](https://mlsanity.com/paper/2309.11691/) (2023)
- [Diversity Drives Fairness: Ensemble of Higher Order Mutants for Intersectional Fairness of Machine Learning Software](https://mlsanity.com/paper/2412.08167/) (2024)
- [Towards Better Fairness-Utility Trade-off: A Comprehensive Measurement-Based Reinforcement Learning Framework](https://mlsanity.com/paper/2307.11379/) (2023)
- [RobustFair: Adversarial Evaluation through Fairness Confusion Directed Gradient Search](https://mlsanity.com/paper/2305.10906/) (2023)
- [Distribution-aware Fairness Test Generation](https://mlsanity.com/paper/2305.13935/) (2023)
- [Search-Based Fairness Testing: An Overview](https://mlsanity.com/paper/2311.06175/) (2023)
- [Perturbation Effects on Robustness and Individual Fairness](https://mlsanity.com/paper/2404.01356/) (2024)
- [Fairpriori: Improving Biased Subgroup Discovery for Deep Neural Network Fairness](https://mlsanity.com/paper/2407.01595/) (2024)
- [Fair Overlap Number of Balls (Fair-ONB): A Data-Morphology-based Undersampling Method for Bias Reduction](https://mlsanity.com/paper/2407.14210/) (2024)
- [TabChange: Precise Attribute Changes in Tabular Data](https://mlsanity.com/paper/2606.00503/) (2026)

## Abstract (arXiv)

Although deep learning has demonstrated astonishing performance in many applications, there are still concerns about its dependability. One desirable property of deep learning applications with societal impact is fairness (i.e., non-discrimination). Unfortunately, discrimination might be intrinsically embedded into the models due to the discrimination in the training data. As a countermeasure, fairness testing systemically identifies discriminatory samples, which can be used to retrain the model and improve the model's fairness. Existing fairness testing approaches however have two major limitations. Firstly, they only work well on traditional machine learning models and have poor performance (e.g., effectiveness and efficiency) on deep learning models. Secondly, they only work on simple structured (e.g., tabular) data and are not applicable for domains such as text. In this work, we bridge the gap by proposing a scalable and effective approach for systematically searching for discriminatory samples while extending existing fairness testing approaches to address a more challenging domain, i.e., text classification. Compared with state-of-the-art methods, our approach only employs lightweight procedures like gradient computation and clustering, which is significantly more scalable and effective. Experimental results show that on average, our approach explores the search space much more effectively (9.62 and 2.38 times more than the state-of-the-art methods respectively on tabular and text datasets) and generates much more discriminatory samples (24.95 and 2.68 times) within a same reasonable time. Moreover, the retrained models reduce discrimination by 57.2% and 60.2% respectively on average.