# Automation Bias in the AI Act: On the Legal Implications of Attempting to De-Bias Human Oversight of AI

arXiv: https://arxiv.org/abs/2502.10036 · HTML: https://mlsanity.com/paper/2502.10036/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2025.

## Built on by (most cited first, 7 of 7)

- [Formalising Human-in-the-Loop: Computational Reductions, Failure Modes, and Legal-Moral Responsibility](https://mlsanity.com/paper/2505.10426/) (2025)
- [Position: Human-Centric AI Requires a Minimum Viable Level of Human Understanding](https://mlsanity.com/paper/2602.00854/) (2026)
- [Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems](https://mlsanity.com/paper/2605.16278/) (2026)
- [Autonomy Reshapes How Personalization Affects Privacy Concerns and Trust in LLM Agents](https://mlsanity.com/paper/2510.04465/) (2025)
- [Gender Bias in LLMs: Preliminary Evidence from Shared Parenting Scenario in Czech Family Law](https://mlsanity.com/paper/2601.05879/) (2026)
- [A Framework for Measuring Appropriate Reliance on Set-Valued AI Advice](https://mlsanity.com/paper/2606.06081/) (2026)
- [Anticipatory Human Oversight of Agentic AI: A Philosophical Account](https://mlsanity.com/paper/2609.24242/) (2026)

## Abstract (arXiv)

This paper examines the legal implications of the explicit mentioning of automation bias (AB) in the Artificial Intelligence Act (AIA). The AIA mandates human oversight for high-risk AI systems and requires providers to enable awareness of AB, i.e., the human tendency to over-rely on AI outputs. The paper analyses the embedding of this extra-juridical concept in the AIA, the asymmetric division of responsibility between AI providers and deployers for mitigating AB, and the challenges of legally enforcing this novel awareness requirement. The analysis shows that the AIA's focus on providers does not adequately address design and context as causes of AB, and questions whether the AIA should directly regulate the risk of AB rather than just mandating awareness. As the AIA's approach requires a balance between legal mandates and behavioural science, the paper proposes that harmonised standards should reference the state of research on AB and human-AI interaction, holding both providers and deployers accountable. Ultimately, further empirical research on human-AI interaction will be essential for effective safeguards.