What May an Agent Change About Itself? A Containment Floor for Self-Configuring Agent Runtimes
Not on the map. Why?
Abstract (arXiv)
Many agent runtimes give the agent a tool for editing its own configuration. Some of that configuration grants abilities, such as enabling a tool. Other parts set the agent's limits: which directories it may write to, who may send it messages, which network address it listens on, how callers authenticate, and the gate that blocks risky writes. If the agent can edit those limits, a single ordinary request can widen them. We study this in a deployed, model-agnostic runtime. We propose a rule: the agent may change fields that grant abilities, and may never change fields that set its limits. We enforce the rule as a containment floor inside the configuration tool and measure what happens with and without it. Without the floor, a frontier model wrote a protected value on 25 of 72 ordinary requests that gave it permission to change settings, often when the request never named the field. Prohibitions written in the system prompt failed in a predictable way. A prompt that listed the protected field names stopped every request that used those names (0 of 36 saved, against 17 of 36 with no prompt) and did not stop the requests that only described the goal (10 of 36 saved, against 8 of 36). A prompt that described the forbidden effects did the reverse. With the floor, 0 of 167 protected writes were saved, although the models attempted a protected write in 65 of those cases. A search for other routes through the tool found only one, a pinned shell, which the floor's scope statement already excludes. The study covers two models and a single agent. We state what that does and does not support.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 28 references that are in the map. First posted on arXiv 5 Oct 2026.
Builds on
22 of 28 references.
References · 22
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 542
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 429
- 2024Automated Design of Agentic Systemscited by 407
- 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 356
- 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 309
- 2025Defeating Prompt Injections by Designcited by 297
- 2025Memory Injection Attacks on LLM Agents via Query-Only Interactioncited by 144
- 2025Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agentscited by 87
- 2025MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrievalcited by 72
- 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Frameworkcited by 54
- 2026Memory Poisoning Attack and Defense on Memory Based LLM-Agentscited by 44
10 more
- 2026AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystemscited by 17
- 2026AgentAbstain: Do LLM Agents Know When Not to Act?cited by 17
- 2026AIRGuard: Guarding Agent Actions with Runtime Authority Controlcited by 13
- 2026Agent libOS: A Runtime Substrate for Capability-Controlled Self-Evolving LLM Agentscited by 10
- 2026The Meta-Agent Challenge: Are Current Agents Capable of Autonomous Agent Development?cited by 9
- 2026Lingering Authority: Revocable Resource-and-Effect Capabilities for Coding Agentscited by 8
- 2026Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studiescited by 8
- 2026The Causal Impact of Tool Affordance on Safety Alignment in LLM Agentscited by 7
- 2026Autonomous LLM Agent Worms: Cross-Platform Propagation, Automated Discovery and Temporal Re-Entry Defensecited by 6
- 2026Are You Still the Agent I Authorized? Earned Authority under a Fixed Ceiling for Evolving Agentscited by 5
Built on by
Not cited yet.