# FairlyUncertain: A Comprehensive Benchmark of Uncertainty in Algorithmic Fairness

arXiv: https://arxiv.org/abs/2410.02005 · HTML: https://mlsanity.com/paper/2410.02005/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 58 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Builds on (most cited first, 30 of 35)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges](https://mlsanity.com/paper/2011.06225/) (2020)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer](https://mlsanity.com/paper/1711.06664/) (2017)
- [When Do Neural Nets Outperform Boosted Trees on Tabular Data?](https://mlsanity.com/paper/2305.02997/) (2023)
- [Fair Regression: Quantitative Definitions and Reduction-based Algorithms](https://mlsanity.com/paper/1905.12843/) (2019)
- [Why Is My Classifier Discriminatory?](https://mlsanity.com/paper/1805.12002/) (2018)
- [Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty](https://mlsanity.com/paper/2011.07586/) (2020)
- [On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks](https://mlsanity.com/paper/2203.09168/) (2022)
- [Machine Learning with a Reject Option: A survey](https://mlsanity.com/paper/2107.11277/) (2021)
- [The Measure and Mismeasure of Fairness](https://mlsanity.com/paper/1808.00023/) (2018)
- [Algorithmic Fairness Datasets: the Story so Far](https://mlsanity.com/paper/2202.01711/) (2022)
- [Faithful Heteroscedastic Regression with Neural Networks](https://mlsanity.com/paper/2212.09184/) (2022)
- [Selective Regression Under Fairness Criteria](https://mlsanity.com/paper/2110.15403/) (2021)
- [UMIX: Improving Importance Weighting for Subpopulation Shift via Uncertainty-Aware Mixup](https://mlsanity.com/paper/2209.08928/) (2022)
- [Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification](https://mlsanity.com/paper/2301.11562/) (2023)
- [Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian Inference](https://mlsanity.com/paper/2010.09851/) (2020)
- [Assessing Fairness in the Presence of Missing Data](https://mlsanity.com/paper/2112.04899/) (2021)
- [Selective Ensembles for Consistent Predictions](https://mlsanity.com/paper/2111.08230/) (2021)
- [Conformalized Fairness via Quantile Regression](https://mlsanity.com/paper/2210.02015/) (2022)
- [Aleatoric and Epistemic Discrimination: Fundamental Limits of Fairness Interventions](https://mlsanity.com/paper/2301.11781/) (2023)

## Built on by (most cited first, 2 of 2)

- [Unequal Uncertainty: Rethinking Algorithmic Interventions for Mitigating Discrimination from AI](https://mlsanity.com/paper/2508.07872/) (2025)
- [Beyond Bias Scores: Answerability Confounds in Activation Steering with Gradient-Constrained Direction Design (GCD)](https://mlsanity.com/paper/2610.17641/) (2026)

## Abstract (arXiv)

Fair predictive algorithms hinge on both equality and trust, yet inherent uncertainty in real-world data challenges our ability to make consistent, fair, and calibrated decisions. While fairly managing predictive error has been extensively explored, some recent work has begun to address the challenge of fairly accounting for irreducible prediction uncertainty. However, a clear taxonomy and well-specified objectives for integrating uncertainty into fairness remains undefined. We address this gap by introducing FairlyUncertain, an axiomatic benchmark for evaluating uncertainty estimates in fairness. Our benchmark posits that fair predictive uncertainty estimates should be consistent across learning pipelines and calibrated to observed randomness. Through extensive experiments on ten popular fairness datasets, our evaluation reveals: (1) A theoretically justified and simple method for estimating uncertainty in binary settings is more consistent and calibrated than prior work; (2) Abstaining from binary predictions, even with improved uncertainty estimates, reduces error but does not alleviate outcome imbalances between demographic groups; (3) Incorporating consistent and calibrated uncertainty estimates in regression tasks improves fairness without any explicit fairness interventions. Additionally, our benchmark package is designed to be extensible and open-source, to grow with the field. By providing a standardized framework for assessing the interplay between uncertainty and fairness, FairlyUncertain paves the way for more equitable and trustworthy machine learning practices.