Persona Jailbreaking in Large Language Models
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
Large Language Models (LLMs) are increasingly deployed in domains such as education, mental health and customer support, where stable and consistent personas are critical for reliability. Yet, existing studies focus on narrative or role-playing tasks and overlook how adversarial conversational history alone can reshape induced personas. Black-box persona manipulation remains unexplored, raising concerns for robustness in realistic interactions. In response, we introduce the task of persona editing, which adversarially steers LLM traits through user-side inputs under a black-box, inference-only setting. To this end, we propose PHISH (Persona Hijacking via Implicit Steering in History), the first framework to expose a new vulnerability in LLM safety that embeds semantically loaded cues into user queries to gradually induce reverse personas. We also define a metric to quantify attack success. Across 3 benchmarks and 8 LLMs, PHISH predictably shifts personas, triggers collateral changes in correlated traits, and exhibits stronger effects in multi-turn settings. In high-risk domains mental health, tutoring, and customer support, PHISH reliably manipulates personas, validated by both human and LLM-as-Judge evaluations. Importantly, PHISH causes only a small reduction in reasoning benchmark performance, leaving overall utility largely intact while still enabling significant persona manipulation. While current guardrails offer partial protection, they remain brittle under sustained attack. Our findings expose new vulnerabilities in personas and highlight the need for context-resilient persona in LLMs. Our codebase and dataset is available at: https://github.com/Jivnesh/PHISH
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 21 of its 51 references that are in the map. First posted on arXiv 23 Jan 2026.
Builds on
21 of 51 references.
Closest ancestors · 5
Same direction, newest first
- 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
- 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
- 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
- 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
Other references · 16
- 2013Intriguing properties of neural networkscited by 5,155
- 2022Red Teaming Language Models with Language Modelscited by 1,026
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
- 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
- 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
- 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
- 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traitscited by 234
- 2024Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalizationcited by 208
- 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 155
- 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
- 2024BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Datacited by 50
4 more
- 2025Can LLM Agents Maintain a Persona in Discourse?cited by 15
- 2023Building Persona Consistent Dialogue Agents with Offline Reinforcement Learningcited by 14
- 2023RealBehavior: A Framework for Faithfully Characterizing Foundation Models' Human-like Behavior Mechanismscited by 5
- 2025CAPE: Context-Aware Personality Evaluation Framework for Large Language Modelscited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2024Who's asking? User personas and the mechanics of latent misalignment
- 2026Disentangling Intent from Role: Adversarial Self-Play for Persona-Invariant Safety Alignment
- 2026From Role Prompt to Infinite Thinking: Exploiting Persona Conditioning for Inference Cost Attacks in LLMs ↗
- 2026Your Language Model Secretly Contains Personality Subnetworks
- 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation
- 2026Persona-Model Collapse in Emergent Misalignment
- 2026Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMs
- 2026Efficient Safety Alignment of Language Models via Latent Personality Traits