# SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment

arXiv: https://arxiv.org/abs/2609.02786 · HTML: https://mlsanity.com/paper/2609.02786/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 29 of its 45 references that are in the map.
- First posted on arXiv 2 Sep 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)
- LLM alignment / RLHF / preference optimisation / [Safety alignment and jailbreak attacks](https://mlsanity.com/alignment/safety-alignment-and-jailbreak-attacks/)
- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 29 of 29)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses](https://mlsanity.com/paper/2604.25850/) (2026)
- [MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory](https://mlsanity.com/paper/2601.03192/) (2026)
- [Self-Harness: Harnesses That Improve Themselves](https://mlsanity.com/paper/2606.09498/) (2026)
- [Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents](https://mlsanity.com/paper/2604.10674/) (2026)
- [AgentDyn: Are Your Agent Security Defenses Deployable in Real-World Dynamic Environments?](https://mlsanity.com/paper/2602.03117/) (2026)
- [Self-Distilled Agentic Reinforcement Learning](https://mlsanity.com/paper/2605.15155/) (2026)
- [AgentDoG: A Diagnostic Guardrail Framework for AI Agent Safety and Security](https://mlsanity.com/paper/2601.18491/) (2026)
- [Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents](https://mlsanity.com/paper/2605.30621/) (2026)
- [OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning](https://mlsanity.com/paper/2606.26790/) (2026)
- [Recursive Harness Self-Improvement](https://mlsanity.com/paper/2607.15524/) (2026)
- [AgentAlign: Navigating Safety Alignment in the Shift from Informative to Agentic Large Language Models](https://mlsanity.com/paper/2505.23020/) (2025)
- [HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systems](https://mlsanity.com/paper/2606.01779/) (2026)
- [ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RL](https://mlsanity.com/paper/2606.01619/) (2026)
- [EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning](https://mlsanity.com/paper/2606.03108/) (2026)
- [EvoRubrics: Dynamic Rubrics as Rewards via Adversarial Co-Evolution for LLM Reinforcement Learning](https://mlsanity.com/paper/2606.23038/) (2026)
- [Harness-Aware Self-Evolving: Co-Evolving Model Weights, Harness, and Task Solutions](https://mlsanity.com/paper/2607.03935/) (2026)
- [Agent Safety Alignment via Reinforcement Learning](https://mlsanity.com/paper/2507.08270/) (2025)
- [Auditing Agent Harness Safety](https://mlsanity.com/paper/2605.14271/) (2026)
- [Reducing the Safety Tax in LLM Safety Alignment with On-Policy Self-Distillation](https://mlsanity.com/paper/2605.15239/) (2026)
- [Meta SecAlign: A Secure Foundation LLM Against Prompt Injection Attacks](https://mlsanity.com/paper/2507.02735/) (2025)
- [On-Policy Self-Evolution via Failure Trajectories for Agentic Safety Alignment](https://mlsanity.com/paper/2605.11882/) (2026)
- [AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Security](https://mlsanity.com/paper/2605.29801/) (2026)
- [GPT-Red: Automated Red Teaming via Self-Play at Scale](https://mlsanity.com/paper/2607.26115/) (2026)
- [SHE: Trajectory-driven Safety Harness Evolution for LLM Agents](https://mlsanity.com/paper/2608.09885/) (2026)
- [SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment](https://mlsanity.com/paper/2606.02530/) (2026)
- [RvB: Automating AI System Hardening via Iterative Red-Blue Games](https://mlsanity.com/paper/2601.19726/) (2026)
- [EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents](https://mlsanity.com/paper/2608.05446/) (2026)

## Built on by (most cited first, 6 of 6)

- [Harness-Zero: Harness Distillation via Agent-as-Harness](https://mlsanity.com/paper/2609.24974/) (2026)
- [CoER: Defending against Adaptive Indirect Prompt Injection via Adversarial Co-Evolution and Refinement](https://mlsanity.com/paper/2609.07529/) (2026)
- [SafeCoEvo: Co-Evolving Safety Harnesses and Guards for LLM Agents at Test-Time](https://mlsanity.com/paper/2609.36580/) (2026)
- [Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluation](https://mlsanity.com/paper/2609.31186/) (2026)
- [RLHarness: Co-evolving Procedural Skills with Reinforcement Learning for Long-horizon Multimodal Reasoning](https://mlsanity.com/paper/2609.32326/) (2026)
- [Compositional Safety Failures in Harness Evolution: Identification and Runtime Monitoring](https://mlsanity.com/paper/2609.33123/) (2026)

## Abstract (arXiv)

The performance of LLM-based agents is jointly shaped by the base model and the harness used when interacting with the environment. This exposes them to safety risks in both harmful final responses and multi-step execution trajectories. Existing safety alignment mechanisms often rely on either external harness updates or policy optimization, yet applying either paradigm in isolation fails to bridge runtime control with intrinsic safety. We propose SafeEvolve, an experience-driven self-evolving framework for agent safety alignment. SafeEvolve leverages safety experience from completed on-policy trajectories to drive a continual loop of harness-policy co-evolution. On the harness side, SafeEvolve converts trajectory-level safety evidence into bounded, component-level updates across safety prompt and hierarchical skills, yielding auditable and reversible harness artifacts. On the policy side, SafeEvolve follows a two-stage SFT-RL paradigm, where harness-use SFT bootstraps the policy to actively leverage evolved harness artifacts, and harness-augmented RL further shapes autonomous safety behaviors during multi-step exploration via verifier-decomposed rewards. Through harness-policy co-evolution, SafeEvolve converts safety experience into an evolved runtime harness and improved policy behavior. Experiments on agentic safety benchmarks show that SafeEvolve achieves a stronger safety-utility tradeoff than existing baselines. For Qwen3.5-4B, SafeEvolve achieves a 3× ASR reduction on AgentDojo while improving benign utility from 59.79% to 61.86%.