# Awareness in Practice: Tensions in Access to Sensitive Attribute Data for Antidiscrimination

arXiv: https://arxiv.org/abs/1912.06171 · HTML: https://mlsanity.com/paper/1912.06171/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2019.

## Built on by (most cited first, 23 of 23)

- [Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support](https://mlsanity.com/paper/2112.05675/) (2021)
- [Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs](https://mlsanity.com/paper/2103.06076/) (2021)
- [Fairness for Unobserved Characteristics: Insights from Technological Impacts on Queer Communities](https://mlsanity.com/paper/2102.04257/) (2021)
- [When Fair Ranking Meets Uncertain Inference](https://mlsanity.com/paper/2105.02091/) (2021)
- [Fairness On The Ground: Applying Algorithmic Fairness Approaches to Production Systems](https://mlsanity.com/paper/2103.06172/) (2021)
- [Fair ranking: a critical review, challenges, and future directions](https://mlsanity.com/paper/2201.12662/) (2022)
- [Controlling Bias Exposure for Fair Interpretable Predictions](https://mlsanity.com/paper/2210.07455/) (2022)
- [Removing biased data to improve fairness and accuracy](https://mlsanity.com/paper/2102.03054/) (2021)
- [Demographic-Reliant Algorithmic Fairness: Characterizing the Risks of Demographic Data Collection in the Pursuit of Fairness](https://mlsanity.com/paper/2205.01038/) (2022)
- [Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach](https://mlsanity.com/paper/2109.08549/) (2021)
- [Equalizing Credit Opportunity in Algorithms: Aligning Algorithmic Fairness Research with U.S. Fair Lending Regulation](https://mlsanity.com/paper/2210.02516/) (2022)
- [Data Representativeness in Accessibility Datasets: A Meta-Analysis](https://mlsanity.com/paper/2207.08037/) (2022)
- [When Fairness Meets Privacy: Fair Classification with Semi-Private Sensitive Attributes](https://mlsanity.com/paper/2207.08336/) (2022)
- [When Fair Classification Meets Noisy Protected Attributes](https://mlsanity.com/paper/2307.03306/) (2023)
- [Fairness without Sensitive Attributes via Knowledge Sharing](https://mlsanity.com/paper/2409.18470/) (2024)
- [Subverting Fair Image Search with Generative Adversarial Perturbations](https://mlsanity.com/paper/2205.02414/) (2022)
- [Identities are not Interchangeable: The Problem of Overgeneralization in Fair Machine Learning](https://mlsanity.com/paper/2505.04038/) (2025)
- ["You Can't Fix What You Can't Measure": Privately Measuring Demographic Performance Disparities in Federated Learning](https://mlsanity.com/paper/2206.12183/) (2022)
- [SoK: What Makes Private Learning Unfair?](https://mlsanity.com/paper/2501.14414/) (2025)
- [Machine Learning practices and infrastructures](https://mlsanity.com/paper/2307.06518/) (2023)
- [An Operational Perspective to Fairness Interventions: Where and How to Intervene](https://mlsanity.com/paper/2302.01574/) (2023)
- [A Comprehensive Review on Understanding the Decentralized and Collaborative Approach in Machine Learning](https://mlsanity.com/paper/2503.09833/) (2025)
- [Five policy uses of algorithmic transparency and explainability](https://mlsanity.com/paper/2302.03080/) (2023)

## Abstract (arXiv)

Organizations cannot address demographic disparities that they cannot see. Recent research on machine learning and fairness has emphasized that awareness of sensitive attributes, such as race and sex, is critical to the development of interventions. However, on the ground, the existence of these data cannot be taken for granted. This paper uses the domains of employment, credit, and healthcare in the United States to surface conditions that have shaped the availability of sensitive attribute data. For each domain, we describe how and when private companies collect or infer sensitive attribute data for antidiscrimination purposes. An inconsistent story emerges: Some companies are required by law to collect sensitive attribute data, while others are prohibited from doing so. Still others, in the absence of legal mandates, have determined that collection and imputation of these data are appropriate to address disparities. This story has important implications for fairness research and its future applications. If companies that mediate access to life opportunities are unable or hesitant to collect or infer sensitive attribute data, then proposed techniques to detect and mitigate bias in machine learning models might never be implemented outside the lab. We conclude that today's legal requirements and corporate practices, while highly inconsistent across domains, offer lessons for how to approach the collection and inference of sensitive data in appropriate circumstances. We urge stakeholders, including machine learning practitioners, to actively help chart a path forward that takes both policy goals and technical needs into account.