# The Flaws of Policies Requiring Human Oversight of Government Algorithms

arXiv: https://arxiv.org/abs/2109.05067 · HTML: https://mlsanity.com/paper/2109.05067/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2021.

## Built on by (most cited first, 30 of 44)

- [Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies](https://mlsanity.com/paper/2112.11471/) (2021)
- [Visibility into AI Agents](https://mlsanity.com/paper/2401.13138/) (2024)
- [Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning](https://mlsanity.com/paper/2202.05402/) (2022)
- [Infrastructure for AI Agents](https://mlsanity.com/paper/2501.10114/) (2025)
- [Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support](https://mlsanity.com/paper/2204.02310/) (2022)
- [Learning When to Advise Human Decision Makers](https://mlsanity.com/paper/2209.13578/) (2022)
- [Intelligent AI Delegation](https://mlsanity.com/paper/2602.11865/) (2026)
- [Imagining new futures beyond predictive systems in child welfare: A qualitative study with impacted stakeholders](https://mlsanity.com/paper/2205.08928/) (2022)
- [Fair When Trained, Unfair When Deployed: Observable Fairness Measures are Unstable in Performative Prediction Settings](https://mlsanity.com/paper/2202.05049/) (2022)
- [A Human-Centric Take on Model Monitoring](https://mlsanity.com/paper/2206.02868/) (2022)
- [Mind the Gap! Bridging Explainable Artificial Intelligence and Human Understanding with Luhmann's Functional Theory of Communication](https://mlsanity.com/paper/2302.03460/) (2023)
- [Predictable Artificial Intelligence](https://mlsanity.com/paper/2310.06167/) (2023)
- [Formalising Human-in-the-Loop: Computational Reductions, Failure Modes, and Legal-Moral Responsibility](https://mlsanity.com/paper/2505.10426/) (2025)
- [Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents](https://mlsanity.com/paper/2606.05391/) (2026)
- [Autonomous Action Runtime Management(AARM):A System Specification for Securing AI-Driven Actions at Runtime](https://mlsanity.com/paper/2602.09433/) (2026)
- [AI Agents Push Humans Out of the Loop](https://mlsanity.com/paper/2608.23642/) (2026)
- [Learning To Guide Human Decision Makers With Vision-Language Models](https://mlsanity.com/paper/2403.16501/) (2024)
- [AI and Human Oversight: A Risk-Based Framework for Alignment](https://mlsanity.com/paper/2510.09090/) (2025)
- [Artificial intelligence in government: Concepts, standards, and a unified framework](https://mlsanity.com/paper/2210.17218/) (2022)
- [The Algorithmic State Architecture (ASA): An Integrated Framework for AI-Enabled Government](https://mlsanity.com/paper/2503.08725/) (2025)
- [Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model](https://mlsanity.com/paper/2608.13867/) (2026)
- [Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems](https://mlsanity.com/paper/2605.16278/) (2026)
- [Lost in Translation: Reimagining the Machine Learning Life Cycle in Education](https://mlsanity.com/paper/2209.03929/) (2022)
- [Improving Task Instructions for Data Annotators: How Clear Rules and Higher Pay Increase Performance in Data Annotation in the AI Economy](https://mlsanity.com/paper/2312.14565/) (2023)
- [An Analysis of the New EU AI Act and A Proposed Standardization Framework for Machine Learning Fairness](https://mlsanity.com/paper/2510.01281/) (2025)
- [Alignment Is Not Enough: A Relational Framework for Moral Standing in Human-AI Interaction](https://mlsanity.com/paper/2603.00078/) (2026)
- [A Framework for Measuring Appropriate Reliance on Set-Valued AI Advice](https://mlsanity.com/paper/2606.06081/) (2026)
- [When Not to Automate: A Formal Protocol for Human Preservation in AI-Optimized Organizations](https://mlsanity.com/paper/2607.15944/) (2026)
- [Systemic Fairness](https://mlsanity.com/paper/2304.06901/) (2023)
- [Guidance Source Matters: How Guidance from AI, Expert, or a Group of Analysts Impacts Visual Data Preparation and Analysis](https://mlsanity.com/paper/2502.00682/) (2025)

## Abstract (arXiv)

As algorithms become an influential component of government decision-making around the world, policymakers have debated how governments can attain the benefits of algorithms while preventing the harms of algorithms. One mechanism that has become a centerpiece of global efforts to regulate government algorithms is to require human oversight of algorithmic decisions. Despite the widespread turn to human oversight, these policies rest on an uninterrogated assumption: that people are able to effectively oversee algorithmic decision-making. In this article, I survey 41 policies that prescribe human oversight of government algorithms and find that they suffer from two significant flaws. First, evidence suggests that people are unable to perform the desired oversight functions. Second, as a result of the first flaw, human oversight policies legitimize government uses of faulty and controversial algorithms without addressing the fundamental issues with these tools. Thus, rather than protect against the potential harms of algorithmic decision-making in government, human oversight policies provide a false sense of security in adopting algorithms and enable vendors and agencies to shirk accountability for algorithmic harms. In light of these flaws, I propose a shift from human oversight to institutional oversight as the central mechanism for regulating government algorithms. This institutional approach operates in two stages. First, agencies must justify that it is appropriate to incorporate an algorithm into decision-making and that any proposed forms of human oversight are supported by empirical evidence. Second, these justifications must receive democratic public review and approval before the agency can adopt the algorithm.