# Anticipatory Human Oversight of Agentic AI: A Philosophical Account

arXiv: https://arxiv.org/abs/2609.24242 · HTML: https://mlsanity.com/paper/2609.24242/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 78 references that are in the map.
- First posted on arXiv 21 Sep 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 16 of 16)

- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [The Ethics of Advanced AI Assistants](https://mlsanity.com/paper/2404.16244/) (2024)
- [Visibility into AI Agents](https://mlsanity.com/paper/2401.13138/) (2024)
- [Fully Autonomous AI Agents Should Not be Developed](https://mlsanity.com/paper/2502.02649/) (2025)
- [The Flaws of Policies Requiring Human Oversight of Government Algorithms](https://mlsanity.com/paper/2109.05067/) (2021)
- [Characterizing AI Agents for Alignment and Governance](https://mlsanity.com/paper/2504.21848/) (2025)
- [On the Quest for Effectiveness in Human Oversight: Interdisciplinary Perspectives](https://mlsanity.com/paper/2404.04059/) (2024)
- [Discerning What Matters: A Multi-Dimensional Assessment of Moral Competence in LLMs](https://mlsanity.com/paper/2506.13082/) (2025)
- [Automation Bias in the AI Act: On the Legal Implications of Attempting to De-Bias Human Oversight of AI](https://mlsanity.com/paper/2502.10036/) (2025)
- [Agentic AI and Multiagentic: Are We Reinventing the Wheel?](https://mlsanity.com/paper/2506.01463/) (2025)
- [Breaking Up with Normatively Monolithic Agency with GRACE: A Reason-Based Neuro-Symbolic Architecture for Safe and Ethical AI Alignment](https://mlsanity.com/paper/2601.10520/) (2026)
- [Disentangling AI Alignment: A Structured Taxonomy Beyond Safety and Ethics](https://mlsanity.com/paper/2506.06286/) (2025)
- [Design Considerations for Human Oversight of AI: Insights from Co-Design Workshops and Work Design Theory](https://mlsanity.com/paper/2510.19512/) (2025)
- [Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems](https://mlsanity.com/paper/2605.16278/) (2026)
- [Constitutive vs. Corrective: A Causal Taxonomy of Human Runtime Involvement in AI Systems](https://mlsanity.com/paper/2603.19213/) (2026)
- [Position: AI Safety Requires Effective Controllability](https://mlsanity.com/paper/2605.27117/) (2026)

## Abstract (arXiv)

Human oversight is widely held to mitigate the risks of AI systems. Even for systems that produce discrete outputs at identifiable decision points, the realisation of human oversight as a reactive measure is empirically fragile, yet increasingly well understood. However, for agentic AI -- systems that plan, decompose goals, and execute multi-step actions over extended horizons -- reactive oversight reaches its structural limits: intervention on individual actions defeats the autonomy that motivates the deployment, while intervention on aggregate patterns is too coarse for harms whose cumulative consequences only become legible after the fact. This paper argues that reactive oversight must be complemented by an anticipatory mode: oversight exercised before the agent acts, by specifying the normative agenda that structures the space of permissible action and refining it iteratively through specification, runtime, and inspection. The two are complements -- the agenda's escalation conditions specify when reactive intervention is invoked. Drawing on Meaningful Human Control, we read anticipatory oversight as the operationalisation of distal-reason tracking. In addition, we argue that the proposed framework yields a specific responsibility architecture by design: occupying the anticipatory mode is the discharge of a role-grounded prospective obligation, and backward-looking responsibility takes the form of strict moral answerability -- rationalistic, relational, and holding regardless of fault, in virtue of the principal's prior opportunity for precaution. We develop bridging failure modes, address objections including moral luck and the illusion of control, and close with regulatory, architectural, and empirical implications