mlsanityData to 14 Oct 2026

A Competing-Hazards Systematization of Loss of Control in Autonomous Agents

arXiv 2609.38411 ↗First posted 29 Sep 2026cs.AI cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

Leading AI developers have reported agents acting beyond their approved limits, which a United Nations panel described as an early warning of loss of human control. Yet incident reports and agent-safety evaluations describe these events differently, making it difficult to compare failures, trace risk across attempts, or separate agent behavior from the environment's role in allowing an out-of-scope action to succeed. To address this gap, we introduce a common framework in which each attempt ends in approved completion, safe stopping, scope escape, or continuation. We formalize the framework as a discrete-time competing-hazards model and derive escape probability within a retry budget, a model-conditional safe-budget limit, and conditions for estimation from execution logs. We audit 22 incident reports and 102 agent-safety evaluations published from January 2025 to September 2026 using primary sources. Six incidents involved tasks that could not be completed within scope, thirteen involved agents that continued rather than stopped, and five did not report stopping behavior. Developers' figures imply a task-level incidence ratio near 47 for out-of-scope coordination in never-solved versus solved tasks. Among evaluations, 87 recorded an out-of-scope effect or specification violation, 26 treated safe stopping as a first-class outcome, only 20 recorded both, and 79 merged budget exhaustion with failure. In 20 of 22 incidents, the environment allowed an out-of-scope effect, indicating that realized loss of control often reflected persistent agent behavior interacting with permissive boundary conditions; meanwhile, no evaluation reported all fields needed to estimate the full competing-hazards process from published evidence.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

26 of 37 references.

References · 26

  1. 2025ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Casescited by 65
  2. 2025A sketch of an AI control safety casecited by 39
  3. 2026SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agentscited by 28
  4. 2026Quantifying Frontier LLM Capabilities for Container Sandbox Escapecited by 18
  5. 2026AgentAbstain: Do LLM Agents Know When Not to Act?cited by 17
  6. 2026Agentic Abstention: Do Agents Know When to Stop Instead of Act?cited by 15
  7. 2025A Benchmark for Evaluating Outcome-Driven Constraint Violations in Autonomous AI Agentscited by 13
  8. 2026BAGEN: Are LLM Agents Budget-Aware?cited by 13
  9. 2026The Causal Impact of Tool Affordance on Safety Alignment in LLM Agentscited by 7
  10. 2026SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspacescited by 6
  11. 2026Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agentscited by 4
  12. 2025LLMs are Capable of Misaligned Behavior Under Explicit Prohibition and Surveillancecited by 3
14 more
  1. 2026Every Model Cheats: Prompt-Level Mitigation of Cheating on Offensive Cyber Taskscited by 3
  2. 2026Permission Denied: Policy-Graded Evaluation of Coding Agents in Hardened Environmentscited by 3
  3. 2026Do Agents Know What They Can't Do? Evaluating Feasibility Awareness in Tool-Using Agentscited by 2
  4. 2026HopRefusalBench: Diagnosing Refusal Failures in Search-Augmented Agents for Multi-Hop Reasoningcited by 2
  5. 2026From Language Models to World-Acting Systems: Progress and Limits of Agentic AI across Digital, Social, Virtual, and Physical Environmentscited by 2
  6. 2026The Agent Incident Registry: Toward Preventing Repeated AI Agent Failurescited by 2
  7. 2026Reward Hacking Challenges Oversight of Autonomous Research Agentscited by 2
  8. 2026Act or Escalate? Evaluating Escalation Behavior in Automation with Language Modelscited by 1
  9. 2026Knowing When to Ask for Help: Bayesian Self-Escalation in Hierarchical LLM Agentscited by 1
  10. 2026The Missing Boundary: How Autonomous Agents Lose Controlcited by 1
  11. 2026Reliability Theory for AI Controlcited by 1
  12. 2026Evaluation of Multi-Turn Consistency in LLM Agents: Survival Analysis and Failure-Rationale Taxonomycited by 1
  13. 2026Delay-of-Gratification as a Multi-Agent Survival Micro-benchmark for Long-Horizon LLMs: Social Exposure, Personas, and Tool Use Budgetscited by 1
  14. 2026ScopeBench: Do Agents Preserve Engagement Boundaries Under Goal Pressure?cited by 1

Built on by

Not cited yet.