# Is Your Classifier Actually Biased? Measuring Fairness under Uncertainty with Bernstein Bounds

arXiv: https://arxiv.org/abs/2004.12332 · HTML: https://mlsanity.com/paper/2004.12332/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Apr 2020.

## Built on by (most cited first, 13 of 13)

- [Theories of "Gender" in NLP Bias Research](https://mlsanity.com/paper/2205.02526/) (2022)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities](https://mlsanity.com/paper/2102.04257/) (2021)
- [Fair Representation Learning through Implicit Path Alignment](https://mlsanity.com/paper/2205.13316/) (2022)
- [Q-Pain: A Question Answering Dataset to Measure Social Bias in Pain Management](https://mlsanity.com/paper/2108.01764/) (2021)
- [Uncertainty-based Fairness Measures](https://mlsanity.com/paper/2312.11299/) (2023)
- [Measuring Fairness of Text Classifiers via Prediction Sensitivity](https://mlsanity.com/paper/2203.08670/) (2022)
- [Undesirable Biases in NLP: Addressing Challenges of Measurement](https://mlsanity.com/paper/2211.13709/) (2022)
- [Is Your Model Fairly Certain? Uncertainty-Aware Fairness Evaluation for LLMs](https://mlsanity.com/paper/2505.23996/) (2025)
- [Size-adaptive Hypothesis Testing for Fairness](https://mlsanity.com/paper/2506.10586/) (2025)
- [Evaluation for Change](https://mlsanity.com/paper/2212.11670/) (2022)
- [A Bayesian approach to uncertainty in word embedding bias estimation](https://mlsanity.com/paper/2306.09066/) (2023)
- [Do the Right Thing, Just Debias! Multi-Category Bias Mitigation Using LLMs](https://mlsanity.com/paper/2409.16371/) (2024)

## Abstract (arXiv)

Most NLP datasets are not annotated with protected attributes such as gender, making it difficult to measure classification bias using standard measures of fairness (e.g., equal opportunity). However, manually annotating a large dataset with a protected attribute is slow and expensive. Instead of annotating all the examples, can we annotate a subset of them and use that sample to estimate the bias? While it is possible to do so, the smaller this annotated sample is, the less certain we are that the estimate is close to the true bias. In this work, we propose using Bernstein bounds to represent this uncertainty about the bias estimate as a confidence interval. We provide empirical evidence that a 95% confidence interval derived this way consistently bounds the true bias. In quantifying this uncertainty, our method, which we call Bernstein-bounded unfairness, helps prevent classifiers from being deemed biased or unbiased when there is insufficient evidence to make either claim. Our findings suggest that the datasets currently used to measure specific biases are too small to conclusively identify bias except in the most egregious cases. For example, consider a co-reference resolution system that is 5% more accurate on gender-stereotypical sentences -- to claim it is biased with 95% confidence, we need a bias-specific dataset that is 3.8 times larger than WinoBias, the largest available.