mlsanityData to 7 Oct 2026

Beware of Your Po! Measuring and Mitigating AI Safety Risks in Role-Play Fine-Tuning of LLMs

arXiv 2502.20968 ↗First posted 28 Feb 2025cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

Role-playing enables large language models (LLMs) to engage users in immersive and personalized interactions, but it also introduces significant safety risks. Existing role-play fine-tuning techniques improve role adaptability but may degrade safety performance, particularly for villainous characters. In this work, we conduct the first comprehensive assessment of role-play fine-tuning risks by training 95 role-specific LLMs using RoleBench. Our experiments reveal that role-play fine-tuning leads to a noticeable decline in safety performance, with safety risks varying based on character traits. To tackle this challenge, we propose Safety-Aware Role-Play Fine-Tuning (SaRFT), a novel method designed to balance role-playing capabilities and safety. Extensive experiments on LLaMA-3-8B-Instruct, Gemma-2-9B-it, and Qwen2.5-7B-Instruct demonstrate that SaRFT consistently outperforms state-of-the-art baselines under both LoRA and full-parameter fine-tuning settings. Our findings highlight the necessity of role-adaptive safety measures and provide insights into mitigating role-specific safety risks in role-playing LLMs.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 33 of its 52 references that are in the map. First posted on arXiv 28 Feb 2025.

Builds on

33 of 52 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  9. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  10. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  11. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  12. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
21 more
  1. 2023Character-LLM: A Trainable Agent for Role-Playingcited by 335
  2. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  3. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  4. 2023InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviewscited by 177
  5. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 126
  6. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 114
  7. 2024Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignmentcited by 104
  8. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82
  9. 2024Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuningcited by 71
  10. 2024SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selectioncited by 64
  11. 2023An Emulator for Fine-Tuning Large Language Models using Small Language Modelscited by 61
  12. 2024Neeko: Leveraging Dynamic LoRA for Efficient Multi-Character Role-Playing Agentcited by 52
  13. 2024MMEvol: Empowering Multimodal Large Language Models with Evol-Instructcited by 42
  14. 2024Compress to Impress: Unleashing the Potential of Compressive Memory in Real-World Long-Term Conversationscited by 36
  15. 2024ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decodingcited by 32
  16. 2024DEEM: Diffusion Models Serve as the Eyes of Large Language Models for Image Perceptioncited by 32
  17. 2024How Well Can LLMs Echo Us? Evaluating AI Chatbots' Role-Play Ability with ECHOcited by 19
  18. 2024Self-Prompt Tuning: Enable Autonomous Role-Playing in LLMscited by 12
  19. 2024Jailbreaking as a Reward Misspecification Problemcited by 11
  20. 2024MINDECHO: Role-Playing Language Agents for Key Opinion Leaderscited by 10
  21. 2023PersonaPKT: Building Personalized Dialogue Agents via Parameter-efficient Knowledge Transfercited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 14.

22 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  2. 2025Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalizationcited by 44
  3. 2025Teaching Language Models to Evolve with Users: Dynamic Profile Modeling for Personalized Alignmentcited by 22
  4. 2025Latent Inter-User Difference Modeling for LLM Personalizationcited by 20
  5. 2026OP-Bench: Benchmarking Over-Personalization for Memory-Augmented Personalized Conversational Agentscited by 18
  6. 2026A Survey on LLM-based Conversational User Simulationcited by 14
  7. 2025Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactionscited by 12
  8. 2025IMPersona: Evaluating Individual Level LM Impersonationcited by 11
  9. 2025Shape it Up! Restoring LLM Safety during Finetuningcited by 11
  10. 2026Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influencecited by 6
  11. 2026When Personalization Legitimizes Risks: Uncovering Safety Vulnerabilities in Personalized Dialogue Agentscited by 5
  12. 2026Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMscited by 5
10 more
  1. 2026Surgery: Mitigating Harmful Fine-Tuning for Large Language Models via Attention Sinkcited by 3
  2. 2026Assessing Risks of Large Language Models in Mental Health Support: A Framework for Automated Clinical AI Red Teamingcited by 3
  3. 2026MHSafeEval: Role-Aware Interaction-Level Evaluation of Mental Health Safety in Large Language Modelscited by 3
  4. 2026Stay in Character, Stay Safe: Dual-Cycle Adversarial Self-Evolution for Safety Role-Playing Agentscited by 2
  5. 2025PD3F: A Pluggable and Dynamic DoS-Defense Framework Against Resource Consumption Attacks Targeting Large Language Modelscited by 1
  6. 2026Beyond Creed: A Non-Identity Safety Condition A Strong Empirical Alternative to Identity Framing in Low-Data LoRA Fine-Tuningcited by 1
  7. 2026AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspectivecited by 1
  8. 2026Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMscited by 1
  9. 2026Do LLM Agents Mirror Socio-Cognitive Effects in Power-Asymmetric Conversations?cited by 0
  10. 2026Defending Against Harmful Supervision Hidden in Benign Samplescited by 0