# Aleatoric and Epistemic Discrimination: Fundamental Limits of Fairness Interventions

arXiv: https://arxiv.org/abs/2301.11781 · HTML: https://mlsanity.com/paper/2301.11781/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 76 references that are in the map.
- First posted on arXiv 27 Jan 2023.

## Builds on (most cited first, 30 of 41)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Multiaccuracy: Black-Box Post-Processing for Fairness in Classification](https://mlsanity.com/paper/1805.12317/) (2018)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [Minimax Pareto Fairness: A Multi Objective Perspective](https://mlsanity.com/paper/2011.01821/) (2020)
- [Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees](https://mlsanity.com/paper/1806.06055/) (2018)
- [Identifying and Correcting Label Bias in Machine Learning](https://mlsanity.com/paper/1901.04966/) (2019)
- [Why Is My Classifier Discriminatory?](https://mlsanity.com/paper/1805.12002/) (2018)
- [Wasserstein Fair Classification](https://mlsanity.com/paper/1907.12059/) (2019)
- [Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey](https://mlsanity.com/paper/2207.07068/) (2022)
- [Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination](https://mlsanity.com/paper/1906.00285/) (2019)
- [Robust Optimization for Fairness with Noisy Protected Groups](https://mlsanity.com/paper/2002.09343/) (2020)
- [Omnipredictors](https://mlsanity.com/paper/2109.05389/) (2021)
- [Leveraging Labeled and Unlabeled Data for Consistent Fair Binary Classification](https://mlsanity.com/paper/1906.05082/) (2019)
- [Recovering from Biased Data: Can Fairness Constraints Improve Accuracy?](https://mlsanity.com/paper/1912.01094/) (2019)
- [Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities](https://mlsanity.com/paper/2102.04257/) (2021)
- [Fairness Evaluation in Presence of Biased Noisy Labels](https://mlsanity.com/paper/2003.13808/) (2020)

## Built on by (most cited first, 19 of 19)

- [Uncertainty-based Fairness Measures](https://mlsanity.com/paper/2312.11299/) (2023)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [U-Fair: Uncertainty-based Multimodal Multitask Learning for Fairer Depression Detection](https://mlsanity.com/paper/2501.09687/) (2025)
- [Demystifying Local and Global Fairness Trade-offs in Federated Learning Using Partial Information Decomposition](https://mlsanity.com/paper/2307.11333/) (2023)
- [Intrinsic Fairness-Accuracy Tradeoffs under Equalized Odds](https://mlsanity.com/paper/2405.07393/) (2024)
- [Foundation Models for Scientific Discovery: From Paradigm Enhancement to Paradigm Transition](https://mlsanity.com/paper/2510.15280/) (2025)
- [A Unified View of Group Fairness Tradeoffs Using Partial Information Decomposition](https://mlsanity.com/paper/2406.04562/) (2024)
- [Efficient Fairness-Performance Pareto Front Computation](https://mlsanity.com/paper/2409.17643/) (2024)
- [Investigating Intersectional Bias in Large Language Models using Confidence Disparities in Coreference Resolution](https://mlsanity.com/paper/2508.07111/) (2025)
- [The Statistical Fairness-Accuracy Frontier](https://mlsanity.com/paper/2508.17622/) (2025)
- [Achievable Fairness on Your Data With Utility Guarantees](https://mlsanity.com/paper/2402.17106/) (2024)
- [FairlyUncertain: A Comprehensive Benchmark of Uncertainty in Algorithmic Fairness](https://mlsanity.com/paper/2410.02005/) (2024)
- [Group-Adaptive Threshold Optimization for Robust AI-Generated Text Detection](https://mlsanity.com/paper/2502.04528/) (2025)
- [An Epistemic and Aleatoric Decomposition of Arbitrariness to Constrain the Set of Good Models](https://mlsanity.com/paper/2302.04525/) (2023)
- [From Seeing it to Experiencing it: Interactive Evaluation of Intersectional Voice Bias in Human-AI Speech Interaction](https://mlsanity.com/paper/2604.13067/) (2026)
- [Unequal Uncertainty: Rethinking Algorithmic Interventions for Mitigating Discrimination from AI](https://mlsanity.com/paper/2508.07872/) (2025)
- [Happiness as a Measure of Fairness](https://mlsanity.com/paper/2511.01069/) (2025)
- [Optimal Fairness under Local Differential Privacy](https://mlsanity.com/paper/2511.16377/) (2025)
- [Fairness Theatre: Evaluating Post-Hoc Fairness Interventions in Vendor-Controlled Early Warning Systems](https://mlsanity.com/paper/2609.38552/) (2026)

## Abstract (arXiv)

Machine learning (ML) models can underperform on certain population groups due to choices made during model development and bias inherent in the data. We categorize sources of discrimination in the ML pipeline into two classes: aleatoric discrimination, which is inherent in the data distribution, and epistemic discrimination, which is due to decisions made during model development. We quantify aleatoric discrimination by determining the performance limits of a model under fairness constraints, assuming perfect knowledge of the data distribution. We demonstrate how to characterize aleatoric discrimination by applying Blackwell's results on comparing statistical experiments. We then quantify epistemic discrimination as the gap between a model's accuracy when fairness constraints are applied and the limit posed by aleatoric discrimination. We apply this approach to benchmark existing fairness interventions and investigate fairness risks in data with missing values. Our results indicate that state-of-the-art fairness interventions are effective at removing epistemic discrimination on standard (overused) tabular datasets. However, when data has missing values, there is still significant room for improvement in handling aleatoric discrimination.