# Formalising Human-in-the-Loop: Computational Reductions, Failure Modes, and Legal-Moral Responsibility

arXiv: https://arxiv.org/abs/2505.10426 · HTML: https://mlsanity.com/paper/2505.10426/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 72 references that are in the map.
- First posted on arXiv 15 May 2025.

## Builds on (most cited first, 15 of 15)

- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Measuring Progress on Scalable Oversight for Large Language Models](https://mlsanity.com/paper/2211.03540/) (2022)
- [Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction](https://mlsanity.com/paper/2210.05791/) (2022)
- [The Flaws of Policies Requiring Human Oversight of Government Algorithms](https://mlsanity.com/paper/2109.05067/) (2021)
- [HAIM-DRL: Enhanced Human-in-the-loop Reinforcement Learning for Safe and Efficient Autonomous Driving](https://mlsanity.com/paper/2401.03160/) (2024)
- [On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives](https://mlsanity.com/paper/2404.04059/) (2024)
- [Human-in-the-loop or AI-in-the-loop? Automate or Collaborate?](https://mlsanity.com/paper/2412.14232/) (2024)
- [Human-AI Interactions in Public Sector Decision-Making: "Automation Bias" and "Selective Adherence" to Algorithmic Advice](https://mlsanity.com/paper/2103.02381/) (2021)
- [System Safety and Artificial Intelligence](https://mlsanity.com/paper/2202.09292/) (2022)
- [Establishing and Evaluating Trustworthy AI: Overview and Research Challenges](https://mlsanity.com/paper/2411.09973/) (2024)
- [Automation Bias in the AI Act: On the Legal Implications of Attempting to De-Bias Human Oversight of AI](https://mlsanity.com/paper/2502.10036/) (2025)
- [Design Patterns for Machine Learning Based Systems with Human-in-the-Loop](https://mlsanity.com/paper/2312.00582/) (2023)
- [Manifesto for the Responsible Development of Mathematical Works -- A Tool for Practitioners and for Management](https://mlsanity.com/paper/2306.09131/) (2023)
- [Integrators at War: Mediating in AI-assisted Resort-to-Force Decisions](https://mlsanity.com/paper/2501.06861/) (2025)
- [Designing for Meaningful Human Control in Military Human-Machine Teams](https://mlsanity.com/paper/2305.11892/) (2023)

## Built on by (most cited first, 8 of 8)

- [AI and Human Oversight: A Risk-Based Framework for Alignment](https://mlsanity.com/paper/2510.09090/) (2025)
- [The Problem of Algorithmic Collisions: Mitigating Unforeseen Risks in a Connected World](https://mlsanity.com/paper/2505.20181/) (2025)
- [From Runtime Records to Legal Findings: An Evidentiary-Adequacy Criterion for Agentic AI Oversight](https://mlsanity.com/paper/2607.00941/) (2026)
- [Making LLMs Reliable When It Matters Most: A Five-Layer Architecture for High-Stakes Decisions](https://mlsanity.com/paper/2511.07669/) (2025)
- [A Hybrid Ai Framework For Strategic Patent Portfolio Pruning: Integrating Learning To-Rank And Market Need Analysis For Technology Transfer Optimization](https://mlsanity.com/paper/2509.00958/) (2025)
- [The SMART+ Framework for AI Systems](https://mlsanity.com/paper/2512.08592/) (2025)
- [From Control to Foresight: Simulation as a New Paradigm for Human-Agent Collaboration](https://mlsanity.com/paper/2603.11677/) (2026)
- [Augment Engineering: A Methodology for Multi-Tool AI Orchestration Across Professional Domains](https://mlsanity.com/paper/2605.26146/) (2026)

## Abstract (arXiv)

We use the notion of oracle machines and reductions from computability theory to formalise different Human-in-the-loop (HITL) setups for AI systems, distinguishing between trivial human monitoring (i.e., total functions), single endpoint human action (i.e., many-one reductions), and highly involved human-AI interaction (i.e., Turing reductions). We then proceed to show that the legal status and safety of different setups vary greatly. We present a taxonomy to categorise HITL failure modes, highlighting the practical limitations of HITL setups. We then identify omissions in UK and EU legal frameworks, which focus on HITL setups that may not always achieve the desired ethical, legal, and sociotechnical outcomes. We suggest areas where the law should recognise the effectiveness of different HITL setups and assign responsibility in these contexts, avoiding human "scapegoating". Our work shows an unavoidable trade-off between attribution of legal responsibility, and technical explainability. Overall, we show how HITL setups involve many technical design decisions, and can be prone to failures out of the humans' control. Our formalisation and taxonomy opens up a new analytic perspective on the challenges in creating HITL setups, helping inform AI developers and lawmakers on designing HITL setups to better achieve their desired outcomes.