mlsanityData to 14 Oct 2026

Persona Attack: Incremental Memory Injection Jailbreak Attack against Large Language Models

arXiv 2606.00150 ↗First posted 29 May 2026cs.CR cs.AIcited by 0

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

As Large Language Models evolve for user convenience, vulnerability to jailbreak attacks continues to be reported despite ongoing efforts in safety training. Traditional jailbreak techniques typically focus on a single prompt injection, neglecting the models' ability to remember the flow of conversation and the user's instructions. In this paper, we propose Persona Attack, a memory injection based jailbreak method that manipulates the model's context window through a step by step approach. Experimental results from applying Persona Attack to several widely used LLMs reveal that, as injections accumulate in memory, models increasingly prioritize these instructions over their internal safety alignment mechanisms. Furthermore, our experiments empirically demonstrate that the attack success rate varies not only according to the memory implementation of the model, but also combinations of instructions and can reach 95% under specific instruction configurations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 19 of its 27 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

19 of 27 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Evolving Security in LLMs: A Study of Jailbreak Attacks and Defenses2 Apr
  2. 2025Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulation11 Mar
  3. 2024Exploiting the Index Gradients for Optimization-Based Jailbreaking on Large Language Models11 Dec
  4. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models21 Feb
  5. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  6. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
  7. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  8. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  10. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
  11. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 8

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  5. 2022Red Teaming Language Models with Language Modelscited by 1,026
  6. 2021Ethical and social risks of harm from Language Modelscited by 966
  7. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 740
  8. 2020Aligning AI With Shared Human Valuescited by 723

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation
  2. 2025Enhancing Jailbreak Attacks on LLMs via Persona Prompts
  3. 2025Rewrite to Jailbreak: Discover Learnable and Transferable Implicit Harmfulness Instruction
  4. 2025Stand on The Shoulders of Giants: Building JailExpert from Previous Attack Experience
  5. 2024SQL Injection Jailbreak: A Structural Disaster of Large Language Models
  6. 2026Evolving Skill-Structured Attack Memory Enhances LLM Jailbreaking
  7. 2024JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs
  8. 2026Exploring and Developing a Pre-Model Safeguard with Draft Models