# Evaluating Fairness of Machine Learning Models Under Uncertain and Incomplete Information

arXiv: https://arxiv.org/abs/2102.08410 · HTML: https://mlsanity.com/paper/2102.08410/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 34 references that are in the map.
- First posted on arXiv 16 Feb 2021.

## Builds on (most cited first, 21 of 21)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Fair Clustering Through Fairlets](https://mlsanity.com/paper/1802.05733/) (2018)
- [Fairness in Learning: Classic and Contextual Bandits](https://mlsanity.com/paper/1605.07139/) (2016)
- [Fairness in Recommendation Ranking through Pairwise Comparisons](https://mlsanity.com/paper/1903.00780/) (2019)
- [Fairness in Reinforcement Learning](https://mlsanity.com/paper/1611.03071/) (2016)
- [Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved](https://mlsanity.com/paper/1811.11154/) (2018)
- [Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination](https://mlsanity.com/paper/1906.00285/) (2019)
- [Ranking with Fairness Constraints](https://mlsanity.com/paper/1704.06840/) (2017)
- [The Price of Fair PCA: One Extra Dimension](https://mlsanity.com/paper/1811.00103/) (2018)
- [Fair k-Center Clustering for Data Summarization](https://mlsanity.com/paper/1901.08628/) (2019)
- [Online Learning with an Unknown Fairness Metric](https://mlsanity.com/paper/1802.06936/) (2018)
- [On the relation between accuracy and fairness in binary classification](https://mlsanity.com/paper/1505.05723/) (2015)
- [Equalized odds postprocessing under imperfect group information](https://mlsanity.com/paper/1906.03284/) (2019)
- [Noise-tolerant fair classification](https://mlsanity.com/paper/1901.10837/) (2019)
- [Transfer of Machine Learning Fairness across Domains](https://mlsanity.com/paper/1906.09688/) (2019)
- [Proxy Fairness](https://mlsanity.com/paper/1806.11212/) (2018)
- [On preserving non-discrimination when combining expert advice](https://mlsanity.com/paper/1810.11829/) (2018)

## Built on by (most cited first, 30 of 33)

- [Leveling Down in Computer Vision: Pareto Inefficiencies in Fair Deep Classifiers](https://mlsanity.com/paper/2203.04913/) (2022)
- [Beyond Images: Label Noise Transition Matrix Estimation for Tasks with Lower-Quality Features](https://mlsanity.com/paper/2202.01273/) (2022)
- [What-is and How-to for Fairness in Machine Learning: A Survey, Reflection, and Perspective](https://mlsanity.com/paper/2206.04101/) (2022)
- [Multiaccurate Proxies for Downstream Fairness](https://mlsanity.com/paper/2107.04423/) (2021)
- [Weak Proxies are Sufficient and Preferable for Fairness with Missing Sensitive Attributes](https://mlsanity.com/paper/2210.03175/) (2022)
- [FRAPPE: A Group Fairness Framework for Post-Processing Everything](https://mlsanity.com/paper/2312.02592/) (2023)
- [Can Less be More? When Increasing-to-Balancing Label Noise Rates Considered Beneficial](https://mlsanity.com/paper/2107.05913/) (2021)
- [Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach](https://mlsanity.com/paper/2109.08549/) (2021)
- [Attribution of Predictive Uncertainties in Classification Models](https://mlsanity.com/paper/2107.08756/) (2021)
- [Measuring Model Fairness under Noisy Covariates: A Theoretical Perspective](https://mlsanity.com/paper/2105.09985/) (2021)
- [Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools](https://mlsanity.com/paper/2309.17337/) (2023)
- [Optimizing Black-box Metrics with Iterative Example Weighting](https://mlsanity.com/paper/2102.09492/) (2021)
- [Fairness without Sensitive Attributes via Knowledge Sharing](https://mlsanity.com/paper/2409.18470/) (2024)
- [Estimating and Controlling for Equalized Odds via Sensitive Attribute Predictors](https://mlsanity.com/paper/2207.12497/) (2022)
- [Fairness and Sequential Decision Making: Limits, Lessons, and Opportunities](https://mlsanity.com/paper/2301.05753/) (2023)
- [Beyond Fairness Metrics: Roadblocks and Challenges for Ethical AI in Practice](https://mlsanity.com/paper/2108.06217/) (2021)
- [Manifestations of Xenophobia in AI Systems](https://mlsanity.com/paper/2212.07877/) (2022)
- [Fairness Under Demographic Scarce Regime](https://mlsanity.com/paper/2307.13081/) (2023)
- [SLIDE: a surrogate fairness constraint to ensure fairness consistency](https://mlsanity.com/paper/2202.03165/) (2022)
- [Distributionally Robust Data Join](https://mlsanity.com/paper/2202.05797/) (2022)
- ["You Can't Fix What You Can't Measure": Privately Measuring Demographic Performance Disparities in Federated Learning](https://mlsanity.com/paper/2206.12183/) (2022)
- [Hyper-parameter Tuning for Fair Classification without Sensitive Attribute Access](https://mlsanity.com/paper/2302.01385/) (2023)
- [Learning With Multi-Group Guarantees For Clusterable Subpopulations](https://mlsanity.com/paper/2410.14588/) (2024)
- [Mitigating Algorithmic Bias with Limited Annotations](https://mlsanity.com/paper/2207.10018/) (2022)
- [Multiaccuracy and Multicalibration via Proxy Groups](https://mlsanity.com/paper/2503.02870/) (2025)
- [On Optimal Steering to Achieve Exact Fairness](https://mlsanity.com/paper/2509.15759/) (2025)
- [Bias Begins with Data: The FairGround Corpus for Robust and Reproducible Research on Algorithmic Fairness](https://mlsanity.com/paper/2510.22363/) (2025)
- [Robust Fair Clustering with Group Membership Uncertainty Sets](https://mlsanity.com/paper/2406.00599/) (2024)
- [Demographic-Agnostic Fairness without Harm](https://mlsanity.com/paper/2509.24077/) (2025)
- [Disparate Conditional Prediction in Multiclass Classifiers](https://mlsanity.com/paper/2206.03234/) (2022)

## Abstract (arXiv)

Training and evaluation of fair classifiers is a challenging problem. This is partly due to the fact that most fairness metrics of interest depend on both the sensitive attribute information and label information of the data points. In many scenarios it is not possible to collect large datasets with such information. An alternate approach that is commonly used is to separately train an attribute classifier on data with sensitive attribute information, and then use it later in the ML pipeline to evaluate the bias of a given classifier. While such decoupling helps alleviate the problem of demographic scarcity, it raises several natural questions such as: how should the attribute classifier be trained?, and how should one use a given attribute classifier for accurate bias estimation? In this work we study this question from both theoretical and empirical perspectives. We first experimentally demonstrate that the test accuracy of the attribute classifier is not always correlated with its effectiveness in bias estimation for a downstream model. In order to further investigate this phenomenon, we analyze an idealized theoretical model and characterize the structure of the optimal classifier. Our analysis has surprising and counter-intuitive implications where in certain regimes one might want to distribute the error of the attribute classifier as unevenly as possible among the different subgroups. Based on our analysis we develop heuristics for both training and using attribute classifiers for bias estimation in the data scarce regime. We empirically demonstrate the effectiveness of our approach on real and simulated data.