# When Fair Classification Meets Noisy Protected Attributes

arXiv: https://arxiv.org/abs/2307.03306 · HTML: https://mlsanity.com/paper/2307.03306/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 69 references that are in the map.
- First posted on arXiv 6 Jul 2023.

## Builds on (most cited first, 30 of 31)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Fairness Without Demographics in Repeated Loss Minimization](https://mlsanity.com/paper/1806.08010/) (2018)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Fairness without Demographics through Adversarially Reweighted Learning](https://mlsanity.com/paper/2006.13114/) (2020)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [Problems with Shapley-value-based explanations as feature importance measures](https://mlsanity.com/paper/2002.11097/) (2020)
- [Fair Regression: Quantitative Definitions and Reduction-based Algorithms](https://mlsanity.com/paper/1905.12843/) (2019)
- [FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age](https://mlsanity.com/paper/1908.04913/) (2019)
- [Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination](https://mlsanity.com/paper/1906.00285/) (2019)
- [Robust Optimization for Fairness with Noisy Protected Groups](https://mlsanity.com/paper/2002.09343/) (2020)
- [Learning Optimal and Fair Decision Trees for Non-Discriminative Decision-Making](https://mlsanity.com/paper/1903.10598/) (2019)
- ["What We Can't Measure, We Can't Understand": Challenges to Demographic Data Procurement in the Pursuit of Fairness](https://mlsanity.com/paper/2011.02282/) (2020)
- [Fair Learning with Private Demographic Data](https://mlsanity.com/paper/2002.11651/) (2020)
- [Noise-tolerant fair classification](https://mlsanity.com/paper/1901.10837/) (2019)
- [Awareness in Practice: Tensions in Access to Sensitive Attribute Data for Antidiscrimination](https://mlsanity.com/paper/1912.06171/) (2019)
- [LiFT: A Scalable Framework for Measuring Fairness in ML Applications](https://mlsanity.com/paper/2008.07433/) (2020)
- [When Fair Ranking Meets Uncertain Inference](https://mlsanity.com/paper/2105.02091/) (2021)
- [FairCanary: Rapid Continuous Explainable Fairness](https://mlsanity.com/paper/2106.07057/) (2021)
- [Detecting race and gender bias in visual representation of AI on web search engines](https://mlsanity.com/paper/2106.14072/) (2021)

## Built on by (most cited first, 6 of 6)

- [ABCFair: an Adaptable Benchmark approach for Comparing Fairness Methods](https://mlsanity.com/paper/2409.16965/) (2024)
- [Preserving AUC Fairness in Learning with Noisy Protected Groups](https://mlsanity.com/paper/2505.18532/) (2025)
- [How Far Can Fairness Constraints Help Recover From Biased Data?](https://mlsanity.com/paper/2312.10396/) (2023)
- [Hidden or Inferred: Fair Learning-To-Rank with Unknown Demographics](https://mlsanity.com/paper/2407.17459/) (2024)
- [Discrimination-free Insurance Pricing with Privatized Sensitive Attributes](https://mlsanity.com/paper/2504.11775/) (2025)
- [What Must a Fairness Audit Report When Demographic Data Is Incomplete?](https://mlsanity.com/paper/2506.23033/) (2025)

## Abstract (arXiv)

The operationalization of algorithmic fairness comes with several practical challenges, not the least of which is the availability or reliability of protected attributes in datasets. In real-world contexts, practical and legal impediments may prevent the collection and use of demographic data, making it difficult to ensure algorithmic fairness. While initial fairness algorithms did not consider these limitations, recent proposals aim to achieve algorithmic fairness in classification by incorporating noisiness in protected attributes or not using protected attributes at all. To the best of our knowledge, this is the first head-to-head study of fair classification algorithms to compare attribute-reliant, noise-tolerant and attribute-blind algorithms along the dual axes of predictivity and fairness. We evaluated these algorithms via case studies on four real-world datasets and synthetic perturbations. Our study reveals that attribute-blind and noise-tolerant fair classifiers can potentially achieve similar level of performance as attribute-reliant algorithms, even when protected attributes are noisy. However, implementing them in practice requires careful nuance. Our study provides insights into the practical implications of using fair classification algorithms in scenarios where protected attributes are noisy or partially available.