# It's complicated. The relationship of algorithmic fairness and non-discrimination provisions for high-risk systems in the EU AI Act

arXiv: https://arxiv.org/abs/2501.12962 · HTML: https://mlsanity.com/paper/2501.12962/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 80 references that are in the map.
- First posted on arXiv 22 Jan 2025.

## Builds on (most cited first, 29 of 29)

- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [AI Alignment: A Comprehensive Survey](https://mlsanity.com/paper/2310.19852/) (2023)
- [Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs](https://mlsanity.com/paper/2311.04892/) (2023)
- [Fairness in Machine Learning: Lessons from Political Philosophy](https://mlsanity.com/paper/1712.03586/) (2017)
- [On the Apparent Conflict Between Individual and Group Fairness](https://mlsanity.com/paper/1912.06883/) (2019)
- [Towards a Critical Race Methodology in Algorithmic Fairness](https://mlsanity.com/paper/1912.03593/) (2019)
- [A Clarification of the Nuances in the Fairness Metrics Landscape](https://mlsanity.com/paper/2106.00467/) (2021)
- [The Measure and Mismeasure of Fairness](https://mlsanity.com/paper/1808.00023/) (2018)
- [Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AI](https://mlsanity.com/paper/2005.05906/) (2020)
- [Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation](https://mlsanity.com/paper/2205.04610/) (2022)
- [Fairness in Large Language Models: A Taxonomic Survey](https://mlsanity.com/paper/2404.01349/) (2024)
- [A Moral Framework for Understanding of Fair ML through Economic Models of Equality of Opportunity](https://mlsanity.com/paper/1809.03400/) (2018)
- [The Unfairness of Fair Machine Learning: Levelling down and strict egalitarianism by default](https://mlsanity.com/paper/2302.02404/) (2023)
- [COMPL-AI Framework: A Technical Interpretation and LLM Benchmarking Suite for the EU Artificial Intelligence Act](https://mlsanity.com/paper/2410.07959/) (2024)
- [Generative AI in EU Law: Liability, Privacy, Intellectual Property, and Cybersecurity](https://mlsanity.com/paper/2401.07348/) (2024)
- [Few-Shot Fairness: Unveiling LLM's Potential for Fairness-Aware Classification](https://mlsanity.com/paper/2402.18502/) (2024)
- [Multi-dimensional discrimination in Law and Machine Learning -- A comparative overview](https://mlsanity.com/paper/2302.05995/) (2023)
- [Algorithmic Unfairness through the Lens of EU Non-Discrimination Law: Or Why the Law is not a Decision Tree](https://mlsanity.com/paper/2305.13938/) (2023)
- [Fair Enough: Standardizing Evaluation and Model Selection for Fairness Research in NLP](https://mlsanity.com/paper/2302.05711/) (2023)
- [Oxford Handbook on AI Ethics Book Chapter on Race and Gender](https://mlsanity.com/paper/1908.06165/) (2019)
- [Fairness in Large Language Models in Three Hours](https://mlsanity.com/paper/2408.00992/) (2024)
- [The Fundamental Rights Impact Assessment (FRIA) in the AI Act: Roots, legal obligations and key elements for a model template](https://mlsanity.com/paper/2411.15149/) (2024)
- [Fairness Hacking: The Malicious Practice of Shrouding Unfairness in Algorithms](https://mlsanity.com/paper/2311.06826/) (2023)
- [Towards Effective Discrimination Testing for Generative AI](https://mlsanity.com/paper/2412.21052/) (2024)
- [Using sensitive data to de-bias AI systems: Article 10(5) of the EU AI Act](https://mlsanity.com/paper/2410.14501/) (2024)
- [Acceptable risks in Europe's proposed AI Act: Reasonableness and other principles for deciding how much risk management is enough](https://mlsanity.com/paper/2308.02047/) (2023)

## Abstract (arXiv)

What constitutes a fair decision? This question is not only difficult for humans but becomes more challenging when Artificial Intelligence (AI) models are used. In light of discriminatory algorithmic behaviors, the EU has recently passed the AI Act, which mandates specific rules for high-risk systems, incorporating both traditional legal non-discrimination regulations and machine learning based algorithmic fairness concepts. This paper aims to bridge these two different concepts in the AI Act through: First, a necessary high-level introduction of both concepts targeting legal and computer science-oriented scholars, and second, an in-depth analysis of the AI Act's relationship between legal non-discrimination regulations and algorithmic fairness. Our analysis reveals three key findings: (1.) Most non-discrimination regulations target only high-risk AI systems. (2.) The regulation of high-risk systems encompasses both data input requirements and output monitoring, though these regulations are partly inconsistent and raise questions of computational feasibility. (3.) Finally, we consider the possible (future) interaction of classical EU non-discrimination law and the AI Act regulations. We recommend developing more specific auditing and testing methodologies for AI systems. This paper aims to serve as a foundation for future interdisciplinary collaboration between legal scholars and computer science-oriented machine learning researchers studying discrimination in AI systems.