# EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents arXiv: https://arxiv.org/abs/2609.05903 ยท HTML: https://mlsanity.com/paper/2609.05903/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. - Builds on 31 of its 33 references that are in the map. - First posted on arXiv 5 Sep 2026. ## Placed in - LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/) - LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/) ## Builds on (most cited first, 30 of 31) - [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022) - [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023) - [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023) - [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022) - [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023) - [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025) - [InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents](https://mlsanity.com/paper/2403.02691/) (2024) - [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024) - [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024) - [StruQ: Defending Against Prompt Injection with Structured Queries](https://mlsanity.com/paper/2402.06363/) (2024) - [Formalizing and Benchmarking Prompt Injection Attacks and Defenses](https://mlsanity.com/paper/2310.12815/) (2023) - [Meta-Harness: End-to-End Optimization of Model Harnesses](https://mlsanity.com/paper/2603.28052/) (2026) - [Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://mlsanity.com/paper/2412.14470/) (2024) - [Defeating Prompt Injections by Design](https://mlsanity.com/paper/2503.18813/) (2025) - [Defending Against Indirect Prompt Injection Attacks With Spotlighting](https://mlsanity.com/paper/2403.14720/) (2024) - [SecAlign: Defending Against Prompt Injection with Preference Optimization](https://mlsanity.com/paper/2410.05451/) (2024) - [Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses](https://mlsanity.com/paper/2604.25850/) (2026) - [Progent: Securing AI Agents with Privilege Control](https://mlsanity.com/paper/2504.11703/) (2025) - [LlamaFirewall: An open source guardrail system for building secure AI agents](https://mlsanity.com/paper/2505.03574/) (2025) - [Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents](https://mlsanity.com/paper/2503.00061/) (2025) - [DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents](https://mlsanity.com/paper/2506.12104/) (2025) - [Natural-Language Agent Harnesses](https://mlsanity.com/paper/2603.25723/) (2026) - [AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?](https://mlsanity.com/paper/2602.03117/) (2026) - [IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents](https://mlsanity.com/paper/2508.15310/) (2025) - [Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents](https://mlsanity.com/paper/2605.30621/) (2026) - [SafeHarness: Lifecycle-Integrated Security Architecture for LLM-based Agent Deployment](https://mlsanity.com/paper/2604.13630/) (2026) - [VeRO: A Harness for Agents to Optimize Agents](https://mlsanity.com/paper/2602.22480/) (2026) - [DecodingTrust-Agent Platform (DTap): A Controllable and Interactive Red-Teaming Platform for AI Agents](https://mlsanity.com/paper/2605.04808/) (2026) - [AgentCanary: A Security Evaluation Framework for Autonomous AI Agents in Real Executable Environments](https://mlsanity.com/paper/2606.10484/) (2026) - [Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks](https://mlsanity.com/paper/2603.30016/) (2026) ## Built on by (most cited first, 2 of 2) - [Bad Genius: Counterfactual-Guided Harness Evolution Beyond Task-Specific Shortcuts](https://mlsanity.com/paper/2609.18366/) (2026) - [SafeCoEvo: Co-Evolving Safety Harnesses and Guards for LLM Agents at Test-Time](https://mlsanity.com/paper/2609.36580/) (2026) ## Abstract (arXiv) Large Language Model (LLM) agents are turning language into real-world effects, making safety necessary against both indirect prompt injections and direct harmful requests. System-level safety harnesses add an enforcement layer beyond model-level defenses, but existing harnesses are usually designed once by experts and applied across heterogeneous models and domains. Effective protection is deployment-dependent: models differ in how much enforcement they need before utility declines, while domains differ in the effects, state, and action sequences that must be governed. A harness that is strict enough for one model may over-block another, and a policy that transfers across domains may miss application-specific safety relations. We present EvoSafeHarness, a safety-specific optimization framework that synthesizes a deployable harness for a frozen model in a target domain. It jointly searches a natural-language policy and executable code logic, guided by model behavior, domain specifications, and fresh-context adversarial review to reject benchmark-specific rules. Across four agent benchmark families, EvoSafeHarness achieves a stronger safety-utility frontier than fixed expert-designed defenses. On DecodingTrust-Agent, it reduces average attack success rate from 45.6% to 10.0% at a 3.3-point utility cost and achieves the best score in 14 of 15 cells. On AgentDojo, it reaches 82.8% utility at 0.0% ASR, twice CaMeL's utility at the same operating point, and transfers unchanged to unseen AgentDyn suites. It also achieves the best score on Agent-SafetyBench for every victim and keeps mean ASR below 20% under adaptive PAIR attacks with a refinement budget of 16. Analysis shows that domain semantics determine which safety relations and trajectory state are needed, while model and runtime behavior determine how and where those relations should be enforced.