mlsanityData to 7 Oct 2026

Disentangling Intent from Role: Adversarial Self-Play for Persona-Invariant Safety Alignment

arXiv 2605.01899 ↗First posted 3 May 2026cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 2 more
Abstract (arXiv)

The growing capabilities of large language models (LLMs) have driven their widespread deployment across diverse domains, even in potentially high-risk scenarios. Despite advances in safety alignment techniques, current models remain vulnerable to emerging persona-based jailbreak attacks. Existing research on persona-based jailbreak has primarily focused on attack iterations, yet it lacks systemic and mechanistic constraints on the defense side. To address this challenge, we propose Persona-Invariant Alignment (PIA), an adversarial self-play framework that achieves co-evolution through Persona Lineage Evolution (PLE) on the attack side and Persona-Invariant Consistency Learning (PICL) on the defense side. Theoretically, PICL is grounded in the structural separation hypothesis, using a unilateral KL-divergence constraint to enable the structural decoupling of safety decisions from persona context, thereby maintaining safe behavior under persona-based jailbreak attacks. Experimental results demonstrate that PLE efficiently explores high-risk persona spaces by leveraging lineage-based credit propagation. Meanwhile, the PICL defense method significantly reduces the Attack Success Rate (ASR) while preserving the model's general capability, thereby validating the superiority and robustness of this alignment paradigm. Codes are available at https://github.com/JiajiaLi-1130/PIA.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 52 of its 66 references that are in the map. First posted on arXiv 3 May 2026.

Builds on

52 of 66 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safety2 Feb
  2. 2025Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models9 Jun
  3. 2025STAIR: Improving Safety Alignment with Introspective Reasoning4 Feb
  4. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference20 Jun
  5. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Models31 May
  6. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models23 Oct
  7. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  8. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Models1 Sep
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  10. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 42

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  6. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  7. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  8. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  9. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  10. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,050
  11. 2020Aligning AI With Shared Human Valuescited by 706
  12. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
30 more
  1. 2019On Variational Bounds of Mutual Informationcited by 550
  2. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  3. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  4. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  5. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  6. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  7. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  8. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  9. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  10. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  11. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 226
  12. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 206
  13. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
  14. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 189
  15. 2023InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviewscited by 177
  16. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 156
  17. 2024CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluationcited by 139
  18. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 124
  19. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  20. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 95
  21. 2024Defending LLMs against Jailbreaking Attacks via Backtranslationcited by 65
  22. 2024RigorLLM: Resilient Guardrails for Large Language Models against Undesired Contentcited by 65
  23. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  24. 2023Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Informationcited by 32
  25. 2024Advancing the Robustness of Large Language Models through Self-Denoised Smoothingcited by 15
  26. 2025"Don't Forget the Teachers": Towards an Educator-Centered Understanding of Harms from Large Language Models in Educationcited by 14
  27. 2025Enhancing Jailbreak Attacks on LLMs via Persona Promptscited by 10
  28. 2024MentalGLM Series: Explainable Large Language Models for Mental Health Analysis on Chinese Social Mediacited by 9
  29. 2025PersonaTeaming: Exploring How Introducing Personas Can Improve Automated AI Red-Teamingcited by 8
  30. 2024SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Modelscited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Stay in Character, Stay Safe: Dual-Cycle Adversarial Self-Evolution for Safety Role-Playing Agents
  2. 2026Persona Attack: Incremental Memory Injection Jailbreak Attack against Large Language Models
  3. 2026Compositional Jailbreaking: An Empirical Analysis of Mutator Chain Interactions in Aligned LLMs
  4. 2026Safety Is Not Universal: The Selective Safety Trap in LLM Alignment
  5. 2026Efficient Safety Alignment of Language Models via Latent Personality Traits
  6. 2025ForgeDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models
  7. 2025Persona Features Control Emergent Misalignment
  8. 2025Scaling Patterns in Adversarial Alignment: Evidence from Multi-LLM Jailbreak Experiments