mlsanityData to 7 Oct 2026

ReSI: Recursive Safety Improvement toward Resistant and Resilient AI

arXiv 2610.12233 ↗First posted 8 Oct 2026cs.CR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recursive self-improvement, the participation of AI systems in improving their own capabilities, is beginning to move from theoretical prospect to practice, posing both challenges and opportunities for safety alignment. Models evolve through frequent updates, and their safety alignment requires continual adaptation to each new checkpoint. Meanwhile, with evolving red-teaming methods exposing new vulnerabilities, safety improvement for each checkpoint needs to mitigate exposed vulnerabilities and generalize to risks not yet revealed. Following R2AI, we term these goals resistance to known threats and resilience to unforeseen risks. Recursive self-improvement, in turn, inspires an approach to both goals: safety alignment could likewise advance through successive rounds of evaluation and update. We therefore introduce ReSI, a recursive safety improvement framework that implements this approach through automated research. In each round, ReSI applies diverse red-teaming methods to identify vulnerabilities in the current target model, develops training recipes, and promotes the update with the largest safety gain among those passing a Pareto gate on capability retention as the next target model. Across four dense and mixture-of-experts models, ReSI matches or exceeds evaluated frontier models on in-distribution and out-of-distribution safety benchmarks, and outperforms alignment baselines on nearly all safety evaluations while largely preserving general capabilities. In particular, ReSI reduces the mean X-Teaming attack success rate across the four models from 86.01% to 31.45%, well below GPT-5.6-Luna's leading frontier result of 56.69%, indicating stronger resilience to attacks unseen during training. These findings support recursive safety improvement as a practical path toward resistant and resilient AI.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 49 of its 77 references that are in the map. First posted on arXiv 8 Oct 2026.

Builds on

49 of 77 references.

References · 49

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  8. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  9. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  10. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  11. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  12. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
37 more
  1. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  2. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  3. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  4. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
  5. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 292
  6. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 206
  7. 2025AIDE: AI-Driven Exploration in the Space of Codecited by 206
  8. 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 128
  9. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
  10. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
  11. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinkingcited by 94
  12. 2025AI-Researcher: Autonomous Scientific Innovationcited by 88
  13. 2026Ministral 3cited by 88
  14. 2024Curiosity-driven Red-teaming for Large Language Modelscited by 83
  15. 2025AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integrationcited by 38
  16. 2025Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Modelscited by 33
  17. 2026Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loopscited by 25
  18. 2026Self-Improvements in Modern Agentic Systems: A Surveycited by 17
  19. 2026MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safetycited by 15
  20. 2025AlphaAlign: Incentivizing Safety Alignment with Extremely Simplified Reinforcement Learningcited by 14
  21. 2025SafeWork-R1: Coevolving Safety and Intelligence under the AI-45circ Lawcited by 13
  22. 2025Large Reasoning Models Learn Better Alignment from Flawed Thinkingcited by 13
  23. 2015From Seed AI to Technological Singularity via Recursively Self-Improving Softwarecited by 9
  24. 2025Safety Alignment of LMs via Non-cooperative Gamescited by 6
  25. 2026Be Your Own Red Teamer: Safety Alignment via Self-Play and Reflective Experience Replaycited by 6
  26. 2026TriPlay-RL: Tri-Role Self-Play Reinforcement Learning for LLM Safety Alignmentcited by 5
  27. 2026DARWIN: Evolving Jailbreak Adversary and Guardrail for LLM Safety Evaluation and Protectioncited by 5
  28. 2026GPT-Red: Automated Red Teaming via Self-Play at Scalecited by 5
  29. 2026TRACE: Task-Aware Adaptive Self-Evolving Agentic Jailbreakingcited by 4
  30. 2026Internalizing Safety Understanding in Large Reasoning Models via Verificationcited by 3
  31. 2026EvoDefense: Co-Evolving Black-Box Defense with Large Language Modelscited by 3
  32. 2026CHASE: Adversarial Red-Blue Teaming for Improving LLM Safety using Reinforcement Learningcited by 3
  33. 2026JailbreakSkill: Scaling Automated Red-Teaming with Reusable and Ever-Evolving Skillscited by 3
  34. 2026Recursive self-improvement of AI research agentscited by 3
  35. 2024Towards AI-45circ Law: A Roadmap to Trustworthy AGIcited by 2
  36. 2025textttR2AI: Towards Resistant and Resilient AI in an Evolving Worldcited by 1
  37. 2026Safin-1: Safety from Within through Memory-Native State Evolutioncited by 1

Built on by

Not cited yet.