Self-Healing Harness for Runtime Oversight of Agent Self-Modification
On the map: LLM agents and tool use → Software engineering coding agents also fits: Agent threats, guardrails and governance
Abstract (arXiv)
LLM agents can change their own future behavior, raising a basic control question of which self-generated changes should be allowed to persist. We formulate this as admission control for self-modification. The agent may propose changes to its operating instructions, while an external runtime gate controls persistence. We implement this principle as a model-agnostic self-healing harness that runs a Detect, Notice, Heal, Validate loop around an otherwise unmodified agent. The agent authors candidate behavioral rules in an external workspace, where they receive provisional execution authority during evaluation and acquire persistent cross-episode authority only after measured improvement on the triggering failure without regression beyond a fixed margin on protected cases. Replay provides matched evidence when available, forward trials provide a weaker fallback, and a corpus-level guard re-tests the accumulated active rule set. Across 16 matched Baseline and Harness runs spanning AppWorld, Terminal-Bench, and τ2-Bench, the gate rejected 383 replay-decided proposals. Of these, 211 (55%) improved their triggering failure while degrading a case that previously worked. This shows that locally beneficial self-modifications can introduce collateral regressions often enough to materially affect gate decisions, providing direct empirical motivation for external admission control. Task-completion score is higher under the Harness in all 16 pairs, with two paired bootstrap intervals excluding zero, while repeated-trial reliability is higher in 12 pairs, tied in 4, and lower in none. Because adaptation modifies the policy-inducing context while leaving model weights fixed, admitted changes remain inspectable, reversible, and compatible with closed-weight models.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 29 references that are in the map. First posted on arXiv 21 Sep 2026.
Builds on
18 of 29 references.
References · 18
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Let's Verify Step by Stepcited by 3,598
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,000
- 2023ExpeL: LLM Agents Are Experiential Learnerscited by 814
- 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
6 more
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
- 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
- 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 128
- 2024AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learningcited by 52
- 2026TRACER: Trajectory Risk Aggregation for Critical Episodes in Agentic Reasoningcited by 9
- 2025Learnable Conformal Prediction with Context-Aware Nonconformity Functions for Robotic Planning and Perceptioncited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025VIGIL: A Reflective Runtime for Self-Healing Agents
- 2026Self-Harness: Harnesses That Improve Themselves
- 2026VERSE: Verified Self-Evolving Optimizer for Agent Harnesses
- 2026EvoUndo: Recoverability-Constrained Self-Evolution for LLM Agent Harnesses
- 2026The Harness as the Only Mutable Surface: Compliance-Bounded Self-Evolution of LLM Agents in Credit Pipelines, with a Measured Admission Gate
- 2026Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses
- 2026EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
- 2026Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents