mlsanityData to 7 Oct 2026

Safety of self-evolving systems

47 papers1 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: safety, attack, attacks, defense, malicious

Key facts. Safety of self-evolving systems: 47 papers on arXiv since 2023 (data through 7 Oct 2026). 4 of its 47 mapped papers (9%) cite “Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections” (2026, arXiv 2602.15654). 3 of its 47 mapped papers (6%) cite “Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models” (2025, arXiv 2506.07468). 2 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

2 new since 17 Sep

  1. SkillPoison: Progressive Skill Poisoning via Successful Experiences
  2. SelfOp: An Optimization Algorithm for Self-Improving Security Agents also fits: Self-evolving LLM agents

Foundations

Most cited by papers here

  1. Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections2026cited by 4 here
  2. Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models2025cited by 3 here
  3. SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models2024cited by 2 here
  4. OEP: Poisoning Self-Evolving LLM Agents via Locally Correct but Non-Transferable Experiences2026cited by 2 here
  5. Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Models2023cited by 1 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2022Large Language Models Can Self-Improvecited by 1 here
  2. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 1 here
  3. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 2 here
  4. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 3 here
  5. 2025Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agentscited by 5 here
  6. 2026Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injectionscited by 4 here
  7. 2026OEP: Poisoning Self-Evolving LLM Agents via Locally Correct but Non-Transferable Experiencescited by 2 here
17 more it builds on, not in this chain
  1. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 2 here
  2. 2023Simple synthetic data reduces sycophancy in large language modelscited by 1 here
  3. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 3 here
  4. 2023Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Modelscited by 1 here
  5. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 2 here
  6. 2024Self-Rewarding Language Modelscited by 1 here
  7. 2024Self-playing Adversarial Language Game Enhances LLM Reasoningcited by 2 here
  8. 2024SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Modelscited by 2 here
  9. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 3 here
  10. 2025Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Modelscited by 3 here
  11. 2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 2 here
  12. 2025A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systemscited by 2 here
  13. 2025EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecyclecited by 2 here
  14. 2026CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verificationcited by 3 here
  15. 2026On Safety Risks in Experience-Driven Self-Evolving Agentscited by 2 here
  16. 2026SkillOpt: Executive Strategy for Self-Evolving Agent Skillscited by 3 here
  17. 2026MUSE-Autoskill: Self-Evolving Agents via Skill Creation, Memory, Management, and Evaluationcited by 2 here

The 24 most cited of 60.

Browse

30 of 47, ranked 6–35

  1. 6Trojan's Whisper: Stealthy Manipulation of OpenClaw through Injected Bootstrapped Guidance20261
  2. 7Membrane: A Self-Evolving Contrastive Safety Memory for LLM Agent Defense20261
  3. 8Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies20261
  4. 9Know Thy Enemy: Securing LLMs Against Prompt Injection via Diverse Data Synthesis and Instruction-Level Chain-of-Thought Learning20260
  5. 10Be Your Own Red Teamer: Safety Alignment via Self-Play and Reflective Experience Replay20260
  6. 11TriPlay-RL: Tri-Role Self-Play Reinforcement Learning for LLM Safety Alignment20260
  7. 12Efficient and Adaptable Detection of Malicious LLM Prompts via Bootstrap Aggregation20260
  8. 13SaFeR-Steer: Evolving Multi-Turn MLLMs via Synthetic Bootstrapping and Feedback Dynamics20260
  9. 14SRTJ: Self-Evolving Rule-Driven Training-Free LLM Jailbreaking20260
  10. 15Disentangling Intent from Role: Adversarial Self-Play for Persona-Invariant Safety Alignment20260
All 30
  1. 16Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization20260
  2. 17Proteus: A Self-Evolving Red Team for Agent Skill Ecosystems20260
  3. 18BYORn: Bootstrap Your Own Responses to Defend Large Vision-Language Models Against Backdoor Attacks20260
  4. 19AgentTrust: A Self-Improving Trust Layer for AI-Agent Actions20260
  5. 20Self-Evolving Just-In-Time Memory for Proactive Embodied Safety20260
  6. 21GPT-Red: Automated Red Teaming via Self-Play at Scale20260
  7. 22When Experience Becomes Instruction: Trajectory Poisoning in Self-Evolving Agent Skill Systems20260
  8. 23Query-Only Backdoor Attacks on Self-Evolving Skills via Trajectory Poisoning20260
  9. 24Yesterday's Shield, Today's Spear: A Self-Evolving Safety Guardrail in Production20260
  10. 25ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents20260
  11. 26Practice Makes Unsafe: Skill Misevolution in Self-Improving LLM Agents20260
  12. 27Beyond Handcrafted Security: Towards Self-Evolving Defense for LLM Agents20260
  13. 28EVOMAL: Self-Poisoning in Self-Evolving Coding Agents20260
  14. 29A Self-Evolving Multi-Agent Framework Defense against LLM Jailbreak Attacks20260
  15. 30SIR: Self-improving Red-teaming for Compute Use Agents20260
  16. 31EvoSkill Injection: Red-Teaming Autonomous Skill Generation and Evolution in Self-Evolving Agents20260
  17. 32Harness-agnostic detection and immunization of reward hacking in self-evolving language models20260
  18. 33Self-Instruct Few-Shot Jailbreaking: Decompose the Attack into Pattern and Behavior Learning20250
  19. 34AgentSGEN: Multi-Agent LLM in the Loop for Semantic Collaboration and GENeration of Synthetic Data20250
  20. 35TRIDENT: Enhancing Large Language Model Safety with Tri-Dimensional Diversified Red-Teaming Data Synthesis20250