# Weak Proxies are Sufficient and Preferable for Fairness with Missing Sensitive Attributes

arXiv: https://arxiv.org/abs/2210.03175 · HTML: https://mlsanity.com/paper/2210.03175/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 89 references that are in the map.
- First posted on arXiv 6 Oct 2022.

## Builds on (most cited first, 30 of 52)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approach](https://mlsanity.com/paper/1609.03683/) (2016)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [Combating noisy labels by agreement: A joint training method with co-regularization](https://mlsanity.com/paper/2003.02752/) (2020)
- [Improving fairness in machine learning systems: What do industry practitioners need?](https://mlsanity.com/paper/1812.05239/) (2018)
- [Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotations](https://mlsanity.com/paper/2110.12088/) (2021)
- [Mitigating Neural Network Overconfidence with Logit Normalization](https://mlsanity.com/paper/2205.09310/) (2022)
- [Are Anchor Points Really Indispensable in Label-Noise Learning?](https://mlsanity.com/paper/1906.00189/) (2019)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Learning with Instance-Dependent Label Noise: A Sample Sieve Approach](https://mlsanity.com/paper/2010.02347/) (2020)
- [Peer Loss Functions: Learning from Noisy Labels without Knowing Noise Rates](https://mlsanity.com/paper/1910.03231/) (2019)
- [Deep Learning with Label Differential Privacy](https://mlsanity.com/paper/2102.06062/) (2021)
- [Optimization with Non-Differentiable Constraints with Applications to Fairness, Recall, Churn, and Other Goals](https://mlsanity.com/paper/1809.04198/) (2018)
- [Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved](https://mlsanity.com/paper/1811.11154/) (2018)
- [A Second-Order Approach to Learning with Instance-Dependent Label Noise](https://mlsanity.com/paper/2012.11854/) (2020)
- [Robust Optimization for Fairness with Noisy Protected Groups](https://mlsanity.com/paper/2002.09343/) (2020)
- [To Smooth or Not? When Label Smoothing Meets Noisy Labels](https://mlsanity.com/paper/2106.04149/) (2021)
- [Understanding artificial intelligence ethics and safety](https://mlsanity.com/paper/1906.05684/) (2019)
- [Fair Classification with Group-Dependent Label Noise](https://mlsanity.com/paper/2011.00379/) (2020)
- [Clusterability as an Alternative to Anchor Points When Learning with Noisy Labels](https://mlsanity.com/paper/2102.05291/) (2021)
- [Fairness via Representation Neutralization](https://mlsanity.com/paper/2106.12674/) (2021)
- [Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support](https://mlsanity.com/paper/2112.05675/) (2021)
- [Open-set Label Noise Can Improve Robustness Against Inherent Label Noise](https://mlsanity.com/paper/2106.10891/) (2021)
- [Detecting Corrupted Labels Without Training a Model to Predict](https://mlsanity.com/paper/2110.06283/) (2021)

## Built on by (most cited first, 17 of 17)

- [Unmasking and Improving Data Credibility: A Study with Datasets for Training Harmless Language Models](https://mlsanity.com/paper/2311.11202/) (2023)
- [Post-hoc Bias Scoring Is Optimal For Fair Classification](https://mlsanity.com/paper/2310.05725/) (2023)
- [Fairness Without Harm: An Influence-Guided Active Sampling Approach](https://mlsanity.com/paper/2402.12789/) (2024)
- [The Importance of Human-Labeled Data in the Era of LLMs](https://mlsanity.com/paper/2306.14910/) (2023)
- [Distributionally Generative Augmentation for Fair Facial Attribute Classification](https://mlsanity.com/paper/2403.06606/) (2024)
- [Fairness without Sensitive Attributes via Knowledge Sharing](https://mlsanity.com/paper/2409.18470/) (2024)
- [Hyper-parameter Tuning for Fair Classification without Sensitive Attribute Access](https://mlsanity.com/paper/2302.01385/) (2023)
- [On the Maximal Local Disparity of Fairness-Aware Classifiers](https://mlsanity.com/paper/2406.03255/) (2024)
- [Multiaccuracy and Multicalibration via Proxy Groups](https://mlsanity.com/paper/2503.02870/) (2025)
- [Towards Harmless Rawlsian Fairness Regardless of Demographic Prior](https://mlsanity.com/paper/2411.02467/) (2024)
- [Constructing Fair Latent Space for Intersection of Fairness and Explainability](https://mlsanity.com/paper/2412.17523/) (2024)
- [Demographic-Agnostic Fairness without Harm](https://mlsanity.com/paper/2509.24077/) (2025)
- [Fairness Risks for Group-conditionally Missing Demographics](https://mlsanity.com/paper/2402.13393/) (2024)
- [Alpha and Prejudice: Improving α-sized Worst-case Fairness via Intrinsic Reweighting](https://mlsanity.com/paper/2411.03068/) (2024)
- [A Critical Review of Predominant Bias in Neural Networks](https://mlsanity.com/paper/2502.11031/) (2025)
- [FedFACT: A Provable Framework for Controllable Group-Fairness Calibration in Federated Learning](https://mlsanity.com/paper/2506.03777/) (2025)
- [Beyond Internal Data: Bounding and Estimating Fairness from Incomplete Data](https://mlsanity.com/paper/2508.13040/) (2025)

## Abstract (arXiv)

Evaluating fairness can be challenging in practice because the sensitive attributes of data are often inaccessible due to privacy constraints. The go-to approach that the industry frequently adopts is using off-the-shelf proxy models to predict the missing sensitive attributes, e.g. Meta [Alao et al., 2021] and Twitter [Belli et al., 2022]. Despite its popularity, there are three important questions unanswered: (1) Is directly using proxies efficacious in measuring fairness? (2) If not, is it possible to accurately evaluate fairness using proxies only? (3) Given the ethical controversy over inferring user private information, is it possible to only use weak (i.e. inaccurate) proxies in order to protect privacy? Our theoretical analyses show that directly using proxy models can give a false sense of (un)fairness. Second, we develop an algorithm that is able to measure fairness (provably) accurately with only three properly identified proxies. Third, we show that our algorithm allows the use of only weak proxies (e.g. with only 68.85%accuracy on COMPAS), adding an extra layer of protection on user privacy. Experiments validate our theoretical analyses and show our algorithm can effectively measure and mitigate bias. Our results imply a set of practical guidelines for practitioners on how to use proxies properly. Code is available at github.com/UCSC-REAL/fair-eval.