# Unlawful Proxy Discrimination: A Framework for Challenging Inherently Discriminatory Algorithms

arXiv: https://arxiv.org/abs/2404.14050 · HTML: https://mlsanity.com/paper/2404.14050/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 52 references that are in the map.
- First posted on arXiv 22 Apr 2024.

## Builds on (most cited first, 13 of 13)

- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [Path-Specific Counterfactual Fairness](https://mlsanity.com/paper/1802.08139/) (2018)
- [Fair Inference On Outcomes](https://mlsanity.com/paper/1705.10378/) (2017)
- [Towards a Critical Race Methodology in Algorithmic Fairness](https://mlsanity.com/paper/1912.03593/) (2019)
- [Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AI](https://mlsanity.com/paper/2005.05906/) (2020)
- [Fairlearn: Assessing and Improving Fairness of AI Systems](https://mlsanity.com/paper/2303.16626/) (2023)
- [The Use and Misuse of Counterfactuals in Ethical Machine Learning](https://mlsanity.com/paper/2102.05085/) (2021)
- [What's Sex Got To Do With Fair Machine Learning?](https://mlsanity.com/paper/2006.01770/) (2020)
- [The four-fifths rule is not disparate impact: a woeful tale of epistemic trespassing in algorithmic fairness](https://mlsanity.com/paper/2202.09519/) (2022)
- [Algorithmic Unfairness through the Lens of EU Non-Discrimination Law: Or Why the Law is not a Decision Tree](https://mlsanity.com/paper/2305.13938/) (2023)
- [What is Proxy Discrimination?](https://mlsanity.com/paper/2205.05265/) (2022)

## Built on by (most cited first, 5 of 5)

- [No Thoughts Just AI: Biased LLM Hiring Recommendations Alter Human Decision Making and Limit Human Autonomy](https://mlsanity.com/paper/2509.04404/) (2025)
- [Engineering the Law-Machine Learning Translation Problem: Developing Legally Aligned Models](https://mlsanity.com/paper/2504.16969/) (2025)
- [Debiasing Alternative Data for Credit Underwriting Using Causal Inference](https://mlsanity.com/paper/2410.22382/) (2024)
- [On Explaining Proxy Discrimination and Unfairness in Individual Decisions Made by AI Systems](https://mlsanity.com/paper/2509.25662/) (2025)
- [Toward Fairness in Machine Learning Models for Predicting Treatment Retention and Premature Discontinuation in Medication for Opioid Use Disorder](https://mlsanity.com/paper/2609.22113/) (2026)

## Abstract (arXiv)

Emerging scholarship suggests that the EU legal concept of direct discrimination - where a person is given different treatment on grounds of a protected characteristic - may apply to various algorithmic decision-making contexts. This has important implications: unlike indirect discrimination, there is generally no 'objective justification' stage in the direct discrimination framework, which means that the deployment of directly discriminatory algorithms will usually be unlawful per se. In this paper, we focus on the most likely candidate for direct discrimination in the algorithmic context, termed inherent direct discrimination, where a proxy is inextricably linked to a protected characteristic. We draw on computer science literature to suggest that, in the algorithmic context, 'treatment on the grounds of' needs to be understood in terms of two steps: proxy capacity and proxy use. Only where both elements can be made out can direct discrimination be said to be `on grounds of' a protected characteristic. We analyse the legal conditions of our proposed proxy capacity and proxy use tests. Based on this analysis, we discuss technical approaches and metrics that could be developed or applied to identify inherent direct discrimination in algorithmic decision-making.