mlsanityData to 7 Oct 2026

The Controllability Trap: A Governance Framework for Military AI Agents

arXiv 2603.03515 ↗First posted 3 Mar 2026cs.CY cs.AIcited by 1

On the map: World models → Latent dynamics models and model-based RL

also in 1 more
Abstract (arXiv)

Agentic AI systems - capable of goal interpretation, world modeling, planning, tool use, long-horizon operation, and autonomous coordination - introduce distinct control failures not addressed by existing safety frameworks. We identify six agentic governance failures tied to these capabilities and show how they erode meaningful human control in military settings. We propose the Agentic Military AI Governance Framework (AMAGF), a measurable architecture structured around three pillars: Preventive Governance (reducing failure likelihood), Detective Governance (real-time detection of control degradation), and Corrective Governance (restoring or safely degrading operations). Its core mechanism, the Control Quality Score (CQS), is a composite real-time metric quantifying human control and enabling graduated responses as control weakens. For each failure type, we define concrete mechanisms, assign responsibilities across five institutional actors, and formalize evaluation metrics. A worked operational scenario illustrates implementation, and we situate the framework within established agent safety literature. We argue that governance must move from a binary conception of control to a continuous model in which control quality is actively measured and managed throughout the operational lifecycle.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 14 of its 21 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

14 of 21 references.

References · 14

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  4. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  5. 2016Concrete Problems in AI Safetycited by 1,582
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  7. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  8. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
  9. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 209
  10. 2023Model evaluation for extreme riskscited by 167
  11. 2023Harms from Increasingly Agentic Algorithmic Systemscited by 134
  12. 2019Optimal Policies Tend to Seek Powercited by 115
2 more
  1. 2016The Off-Switch Gamecited by 97
  2. 2023Evaluating Language-Model Agents on Realistic Autonomous Taskscited by 94

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Testing and Evaluation of Agentic AI Systems In Military Command and Control
  2. 2026Beyond Task Success: An Evidence-Synthesis Framework for Evaluating, Governing, and Orchestrating Agentic AI
  3. 2026The CASE Framework: A Multi-Disciplinary Control Architecture for Governing Enterprise Agentic AI
  4. 2025MI9: An Integrated Runtime Governance Framework for Agentic AI
  5. 2026Autonomy and Agency in Agentic AI: Architectural Tactics for Regulated Contexts
  6. 2026Beyond single-channel agentic benchmarking
  7. 2026Position: AI Safety Requires Effective Controllability
  8. 2026Exploring Systems-Thinking Approaches to Loss of Control Risk