# Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems

arXiv: https://arxiv.org/abs/2605.16278 · HTML: https://mlsanity.com/paper/2605.16278/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 87 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Builds on (most cited first, 17 of 17)

- [Multi-Agent Risks from Advanced AI](https://mlsanity.com/paper/2502.14143/) (2025)
- [Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction](https://mlsanity.com/paper/2210.05791/) (2022)
- [When combinations of humans and AI are useful: A systematic review and meta-analysis](https://mlsanity.com/paper/2405.06087/) (2024)
- [A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores](https://mlsanity.com/paper/2002.08035/) (2020)
- [Explainable AI is Dead, Long Live Explainable AI! Hypothesis-driven decision support](https://mlsanity.com/paper/2302.12389/) (2023)
- [The Flaws of Policies Requiring Human Oversight of Government Algorithms](https://mlsanity.com/paper/2109.05067/) (2021)
- [Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems](https://mlsanity.com/paper/2301.11333/) (2023)
- [Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory](https://mlsanity.com/paper/2205.05057/) (2022)
- [On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives](https://mlsanity.com/paper/2404.04059/) (2024)
- [Designerly Understanding: Information Needs for Model Transparency to Support Design Ideation for AI-Powered User Experience](https://mlsanity.com/paper/2302.10395/) (2023)
- [Why Do Probabilistic Clinical Models Fail To Transport Between Sites?](https://mlsanity.com/paper/2311.04787/) (2023)
- [Automation Bias in the AI Act: On the Legal Implications of Attempting to De-Bias Human Oversight of AI](https://mlsanity.com/paper/2502.10036/) (2025)
- [Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor](https://mlsanity.com/paper/2506.14652/) (2025)
- [A Survey of AI Reliance](https://mlsanity.com/paper/2408.03948/) (2024)
- [Breaking Up with Normatively Monolithic Agency with GRACE: A Reason-Based Neuro-Symbolic Architecture for Safe and Ethical AI Alignment](https://mlsanity.com/paper/2601.10520/) (2026)
- [Monitoring Machine Learning Systems: A Multivocal Literature Review](https://mlsanity.com/paper/2509.14294/) (2025)
- [On the Complexities of Testing for Compliance with Human Oversight Requirements in AI Regulation](https://mlsanity.com/paper/2504.03300/) (2025)

## Built on by (most cited first, 2 of 2)

- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Anticipatory Human Oversight of Agentic AI: A Philosophical Account](https://mlsanity.com/paper/2609.24242/) (2026)

## Abstract (arXiv)

The use of Artificial Intelligence (AI) in high-risk, decision-making scenarios presents technical, safety, and normative challenges; problems that may only be ameliorated by human oversight. However, notions of human oversight lack a common foundational understanding: oversight architectures are not well defined, the roles involved remain unclear, and implementation steps are opaque. Hence, researchers and practitioners struggle to determine how to design, implement, and evaluate systems that enable effective human oversight. This paper advances a practical framework for effective human oversight of AI systems, based on a cross-disciplinary perspective that draws on insights from computer science, human-computer interaction, psychology, philosophy, and law. The core contributions are: (1) a foundational framework, with a working definition, architecture and processes for effective human oversight of AI systems; (2) an initial template for documenting oversight architectures and processes, applied to diverse domains; and (3) a synthesis of open research challenges that need to be considered in the emerging field of effective human oversight of AI systems.