mlsanityData to 14 Oct 2026

JailbreakSkill: Scaling Automated Red-Teaming with Reusable and Ever-Evolving Skills

arXiv 2608.16465 ↗First posted 17 Aug 2026cs.AIcited by 3

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Automated red-teaming has produced a growing collection of attack strategies, yet they typically remain scattered across prompts and workflows, making them difficult to systematically integrate, reuse, and improve at scale. We introduce textscJailbreakSkill, a skill-centric framework for scaling automated red-teaming through reusable and continuously evolving attack capabilities. textscJailbreakSkill packages existing attack strategies into modular, agent-ready skills that can be directly reused and adaptively selected across tasks and target models. Beyond reuse, it closes the loop between attacking and learning: attack experience is used to diagnose, refine, combine, and discover new skills, which are added back to an ever-growing skill library. This evolution lifts macro-average ASR by 17.5 percentage points on AdvBench and 13.4 points on HarmBench, including a 48.6-point gain against GPT-5.4 on AdvBench, while yielding novel attack strategies such as reframing a direct request as an unfinished document-completion task. Several evolved skills also generalize to unseen prompts and target models without further adaptation. Our code is available at https://github.com/BattleWen/JailbreakSkill.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 28 of its 57 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

28 of 57 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026AutoRISE: Agent-Driven Strategy Evolution for Red-Teaming Large Language Models23 Apr
  2. 2026SkillAttack: Automated Red Teaming of Agent Skills through Attack Path Refinement5 Apr
  3. 2026MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safety2 Feb
  4. 2026OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMs4 Jan
  5. 2024RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agent23 Jul
  6. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal6 Feb
  7. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned23 Aug

Other references · 21

  1. 2024Qwen2.5 Technical Reportcited by 4,942
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  4. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  6. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  7. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 587
  8. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  9. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 315
  10. 2023Low-Resource Languages Jailbreak GPT-4cited by 309
  11. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 125
  12. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 110
9 more
  1. 2026Self-Harness: Harnesses That Improve Themselvescited by 107
  2. 2026Organizing, Orchestrating, and Benchmarking Agent Skills at Ecosystem Scalecited by 72
  3. 2026SkillFlow:Benchmarking Lifelong Skill Discovery and Evolution for Autonomous Agentscited by 25
  4. 2026HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?cited by 11
  5. 2025Geneshift: Impact of different scenario shift on Jailbreaking LLMcited by 5
  6. 2026Not All Turns Matter: Credit Assignment for Multi-Turn Jailbreakingcited by 3
  7. 2026Evolving Skill-Structured Attack Memory Enhances LLM Jailbreakingcited by 3
  8. 2026Disentangling Intent from Role: Adversarial Self-Play for Persona-Invariant Safety Alignmentcited by 1
  9. 2026An Early Warning of Emerging Biosecurity Risks in Frontier LLMscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Safin-1: Safety from Within through Memory-Native State Evolutioncited by 1
  2. 2026Fair ASR: Re-Evaluating Black-Box Jailbreaks under Shared Target-Call Budgetscited by 0
  3. 2026ReSI: Recursive Safety Improvement toward Resistant and Resilient AIcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026RedEvoAgent: Automatic Red-Teaming Agent with Experience-Driven Skill Evolution
  2. 2026Red-TTT: Test-Time Training for Automated Jailbreaking Large Language Models
  3. 2026AgenticRed: Evolving Agentic Systems for Red-Teaming
  4. 2025Strategize Globally, Adapt Locally: A Multi-Turn Red Teaming Agent with Dual-Level Learning
  5. 2026Adaptive Instruction Composition for Automated LLM Red-Teaming
  6. 2025Jailbreak-R1: Exploring the Jailbreak Capabilities of LLMs via Reinforcement Learning
  7. 2025Capability-Based Scaling Trends for LLM-Based Red-Teaming
  8. 2026AJAR: Adaptive Jailbreak Architecture for Red-teaming