# SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models

arXiv: https://arxiv.org/abs/2408.02632 · HTML: https://mlsanity.com/paper/2408.02632/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Safety of self-evolving systems, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 35 of its 51 references that are in the map.
- First posted on arXiv 5 Aug 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [Safety of self-evolving systems](https://mlsanity.com/synth/safety-of-self-evolving-systems/)
- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 30 of 35)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://mlsanity.com/paper/2307.04657/) (2023)
- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts](https://mlsanity.com/paper/2309.10253/) (2023)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation](https://mlsanity.com/paper/2106.04399/) (2021)
- [Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions](https://mlsanity.com/paper/2309.07875/) (2023)
- [DeepInception: Hypnotize Large Language Model to Be Jailbreaker](https://mlsanity.com/paper/2311.03191/) (2023)
- [We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?](https://mlsanity.com/paper/2407.01284/) (2024)
- [Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment](https://mlsanity.com/paper/2308.09662/) (2023)
- [Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts](https://mlsanity.com/paper/2402.16822/) (2024)
- [MART: Improving LLM Safety with Multi-round Automatic Red-Teaming](https://mlsanity.com/paper/2311.07689/) (2023)
- [ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming](https://mlsanity.com/paper/2404.08676/) (2024)

## Built on by (most cited first, 9 of 9)

- [OJBench: A Competition Level Code Benchmark For Large Language Models](https://mlsanity.com/paper/2506.16395/) (2025)
- [Towards Robust Multimodal Large Language Models Against Jailbreak Attacks](https://mlsanity.com/paper/2502.00653/) (2025)
- [Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directions](https://mlsanity.com/paper/2509.25539/) (2025)
- [Self-evolving Embodied AI](https://mlsanity.com/paper/2602.04411/) (2026)
- [Disentangling Intent from Role: Adversarial Self-Play for Persona-Invariant Safety Alignment](https://mlsanity.com/paper/2605.01899/) (2026)
- [EvoJail: Evolutionary Diverse Jailbreak Prompt Generation for Large Language Models](https://mlsanity.com/paper/2605.02921/) (2026)
- [Decoupling Safety into Orthogonal Subspace: Cost-Efficient and Performance-Preserving Alignment for Large Language Models](https://mlsanity.com/paper/2510.09004/) (2025)
- [XekRung Technical Report](https://mlsanity.com/paper/2605.00072/) (2026)
- [AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety](https://mlsanity.com/paper/2610.19422/) (2026)

## Abstract (arXiv)

As large language models (LLMs) continue to advance in capability and influence, ensuring their security and preventing harmful outputs has become crucial. A promising approach to address these concerns involves training models to automatically generate adversarial prompts for red teaming. However, the evolving subtlety of vulnerabilities in LLMs challenges the effectiveness of current adversarial methods, which struggle to specifically target and explore the weaknesses of these models. To tackle these challenges, we introduce the Self-Evolving Adversarial Safety (SEAS) optimization framework, which enhances security by leveraging data generated by the model itself. SEAS operates through three iterative stages: Initialization, Attack, and Adversarial Optimization, refining both the Red Team and Target models to improve robustness and safety. This framework reduces reliance on manual testing and significantly enhances the security capabilities of LLMs. Our contributions include a novel adversarial framework, a comprehensive safety dataset, and after three iterations, the Target model achieves a security level comparable to GPT-4, while the Red Team model shows a marked increase in attack success rate (ASR) against advanced models. Our code and datasets are released at https://SEAS-LLM.github.io/.