# Systematic Evaluation of Predictive Fairness

arXiv: https://arxiv.org/abs/2210.08758 · HTML: https://mlsanity.com/paper/2210.08758/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 48 references that are in the map.
- First posted on arXiv 17 Oct 2022.

## Builds on (most cited first, 30 of 31)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019)
- [Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them](https://mlsanity.com/paper/1903.03862/) (2019)
- [Measuring and Reducing Gendered Correlations in Pre-trained Models](https://mlsanity.com/paper/2010.06032/) (2020)
- [Demographic Dialectal Variation in Social Media: A Case Study of African-American English](https://mlsanity.com/paper/1608.08868/) (2016)
- [Adversarial Removal of Demographic Attributes from Text Data](https://mlsanity.com/paper/1808.06640/) (2018)
- [Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies](https://mlsanity.com/paper/2108.12084/) (2021)
- [Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm](https://mlsanity.com/paper/1708.00524/) (2017)
- [Linear Adversarial Concept Erasure](https://mlsanity.com/paper/2201.12091/) (2022)
- [FairBatch: Batch Selection for Model Fairness](https://mlsanity.com/paper/2012.01696/) (2020)
- [An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models](https://mlsanity.com/paper/2110.08527/) (2021)
- [Conditional Learning of Fair Representations](https://mlsanity.com/paper/1910.07162/) (2019)
- [Towards Robust and Privacy-preserving Text Representations](https://mlsanity.com/paper/1805.06093/) (2018)
- [Training individually fair ML models with Sensitive Subspace Robustness](https://mlsanity.com/paper/1907.00020/) (2019)
- [FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing](https://mlsanity.com/paper/2203.07228/) (2022)
- [Diverse Adversaries for Mitigating Bias in Training](https://mlsanity.com/paper/2101.10001/) (2021)
- [Average Individual Fairness: Algorithms, Generalization and Experiments](https://mlsanity.com/paper/1905.10607/) (2019)
- [Your fairness may vary: Pretrained language model fairness in toxic text classification](https://mlsanity.com/paper/2108.01250/) (2021)
- [Evaluating Debiasing Techniques for Intersectional Biases](https://mlsanity.com/paper/2109.10441/) (2021)
- [Optimising Equal Opportunity Fairness in Model Training](https://mlsanity.com/paper/2205.02393/) (2022)
- [Fairness-aware Class Imbalanced Learning](https://mlsanity.com/paper/2109.10444/) (2021)
- [An Empirical Comparison of Bias Reduction Methods on Real-World Problems in High-Stakes Policy Settings](https://mlsanity.com/paper/2105.06442/) (2021)
- [Towards Equal Opportunity Fairness through Adversarial Learning](https://mlsanity.com/paper/2203.06317/) (2022)

## Built on by (most cited first, 3 of 3)

- [Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMs](https://mlsanity.com/paper/2308.13387/) (2023)
- [AI Safety in Generative AI Large Language Models: A Survey](https://mlsanity.com/paper/2407.18369/) (2024)
- [CHiSafetyBench: A Chinese Hierarchical Safety Benchmark for Large Language Models](https://mlsanity.com/paper/2406.10311/) (2024)

## Abstract (arXiv)

Mitigating bias in training on biased datasets is an important open problem. Several techniques have been proposed, however the typical evaluation regime is very limited, considering very narrow data conditions. For instance, the effect of target class imbalance and stereotyping is under-studied. To address this gap, we examine the performance of various debiasing methods across multiple tasks, spanning binary classification (Twitter sentiment), multi-class classification (profession prediction), and regression (valence prediction). Through extensive experimentation, we find that data conditions have a strong influence on relative model performance, and that general conclusions cannot be drawn about method efficacy when evaluating only on standard datasets, as is current practice in fairness research.