# Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination

arXiv: https://arxiv.org/abs/1906.00285 · HTML: https://mlsanity.com/paper/1906.00285/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2019.

## Built on by (most cited first, 30 of 79)

- [Fast Model Debias with Machine Unlearning](https://mlsanity.com/paper/2310.12560/) (2023)
- [Learning Fair Classifiers with Partially Annotated Group Labels](https://mlsanity.com/paper/2111.14581/) (2021)
- [Evaluating Fairness of Machine Learning Models Under Uncertain and Incomplete Information](https://mlsanity.com/paper/2102.08410/) (2021)
- [Doubly-Valid/Doubly-Sharp Sensitivity Analysis for Causal Inference with Unmeasured Confounding](https://mlsanity.com/paper/2112.11449/) (2021)
- [Privacy and Fairness in Federated Learning: on the Perspective of Trade-off](https://mlsanity.com/paper/2306.14123/) (2023)
- [Fairness without Imputation: A Decision Tree Approach for Fair Prediction with Missing Values](https://mlsanity.com/paper/2109.10431/) (2021)
- [Fair Classification with Adversarial Perturbations](https://mlsanity.com/paper/2106.05964/) (2021)
- [The Impossibility of Fair LLMs](https://mlsanity.com/paper/2406.03198/) (2024)
- [Algorithm Fairness in AI for Medicine and Healthcare](https://mlsanity.com/paper/2110.00603/) (2021)
- [Aleatoric and Epistemic Discrimination: Fundamental Limits of Fairness Interventions](https://mlsanity.com/paper/2301.11781/) (2023)
- [Fairness-Aware PAC Learning from Corrupted Data](https://mlsanity.com/paper/2102.06004/) (2021)
- [Algorithmic Fairness in Business Analytics: Directions for Research and Practice](https://mlsanity.com/paper/2207.10991/) (2022)
- [Removing biased data to improve fairness and accuracy](https://mlsanity.com/paper/2102.03054/) (2021)
- [Multiaccurate Proxies for Downstream Fairness](https://mlsanity.com/paper/2107.04423/) (2021)
- [Fair Ranking with Noisy Protected Attributes](https://mlsanity.com/paper/2211.17067/) (2022)
- [Fair Sequential Selection Using Supervised Learning Models](https://mlsanity.com/paper/2110.13986/) (2021)
- [FRAPPE: A Group Fairness Framework for Post-Processing Everything](https://mlsanity.com/paper/2312.02592/) (2023)
- [What's the Harm? Sharp Bounds on the Fraction Negatively Affected by Treatment](https://mlsanity.com/paper/2205.10327/) (2022)
- [Estimation of Fair Ranking Metrics with Incomplete Judgments](https://mlsanity.com/paper/2108.05152/) (2021)
- [Treatment Effect Risk: Bounds and Inference](https://mlsanity.com/paper/2201.05893/) (2022)
- [Inference on Strongly Identified Functionals of Weakly Identified Functions](https://mlsanity.com/paper/2208.08291/) (2022)
- [Fair Off-Policy Learning from Observational Data](https://mlsanity.com/paper/2303.08516/) (2023)
- [FUGNN: Harmonizing Fairness and Utility in Graph Neural Networks](https://mlsanity.com/paper/2405.17034/) (2024)
- [Fairness-Aware Unsupervised Feature Selection](https://mlsanity.com/paper/2106.02216/) (2021)
- [Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach](https://mlsanity.com/paper/2109.08549/) (2021)
- [Fair admission risk prediction with proportional multicalibration](https://mlsanity.com/paper/2209.14613/) (2022)
- [Can Querying for Bias Leak Protected Attributes? Achieving Privacy With Smooth Sensitivity](https://mlsanity.com/paper/2211.02139/) (2022)
- [The Fragility of Fairness: Causal Sensitivity Analysis for Fair Machine Learning](https://mlsanity.com/paper/2410.09600/) (2024)
- [Measuring Model Fairness under Noisy Covariates: A Theoretical Perspective](https://mlsanity.com/paper/2105.09985/) (2021)
- [Fairness-aware Online Price Discrimination with Nonparametric Demand Models](https://mlsanity.com/paper/2111.08221/) (2021)

## Abstract (arXiv)

The increasing impact of algorithmic decisions on people's lives compels us to scrutinize their fairness and, in particular, the disparate impacts that ostensibly-color-blind algorithms can have on different groups. Examples include credit decisioning, hiring, advertising, criminal justice, personalized medicine, and targeted policymaking, where in some cases legislative or regulatory frameworks for fairness exist and define specific protected classes. In this paper we study a fundamental challenge to assessing disparate impacts in practice: protected class membership is often not observed in the data. This is particularly a problem in lending and healthcare. We consider the use of an auxiliary dataset, such as the US census, to construct models that predict the protected class from proxy variables, such as surname and geolocation. We show that even with such data, a variety of common disparity measures are generally unidentifiable, providing a new perspective on the documented biases of popular proxy-based methods. We provide exact characterizations of the tightest-possible set of all possible true disparities that are consistent with the data (and possibly any assumptions). We further provide optimization-based algorithms for computing and visualizing these sets and statistical tools to assess sampling uncertainty. Together, these enable reliable and robust assessments of disparities -- an important tool when disparity assessment can have far-reaching policy implications. We demonstrate this in two case studies with real data: mortgage lending and personalized medicine dosing.