mlsanityData to 14 Oct 2026

Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers

arXiv 2507.13474 ↗First posted 17 Jul 2025cs.CLcited by 3

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

The safety of large language models (LLMs) has garnered significant research attention. In this paper, we argue that previous empirical studies demonstrate LLMs exhibit a propensity to trust information from authoritative sources, such as academic papers, implying new possible vulnerabilities. To verify this possibility, a preliminary analysis is designed to illustrate our two findings. Based on this insight, a novel jailbreaking method, Paper Summary Attack (llmnamePSA), is proposed. It systematically synthesizes content from either attack-focused or defense-focused LLM safety paper to construct an adversarial prompt template, while strategically infilling harmful query as adversarial payloads within predefined subsections. Extensive experiments show significant vulnerabilities not only in base LLMs, but also in state-of-the-art reasoning model like Deepseek-R1. PSA achieves a 97% attack success rate (ASR) on well-aligned models like Claude3.5-Sonnet and an even higher 98% ASR on Deepseek-R1. More intriguingly, our work has further revealed diametrically opposed vulnerability bias across different base models, and even between different versions of the same model, when exposed to either attack-focused or defense-focused papers. This phenomenon potentially indicates future research clues for both adversarial methodologies and safety alignment.Code is available at https://github.com/233liang/Paper-Summary-Attack

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 24 of its 37 references that are in the map. First posted on arXiv 17 Jul 2025.

Builds on

24 of 37 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey5 Jul
  2. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden States9 Jun
  3. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction28 Feb
  4. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  5. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  7. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  8. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM18 Sep
  9. 2023Detecting Language Model Attacks with Perplexity27 Aug
  10. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 13

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2017Deep reinforcement learning from human preferencescited by 3,912
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  6. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  8. 2021Ethical and social risks of harm from Language Modelscited by 966
  9. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 216
  10. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  11. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 163
  12. 2022CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Modelscited by 53
1 more
  1. 2023Influence of External Information on Large Language Models Mirrors Social Cognitive Patternscited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025A Survey on Agentic Security: Applications, Threats and Defensescited by 20
  2. 2026ICON: Intent-Context Coupling for Efficient Multi-Turn Jailbreak Attackcited by 6
  3. 2025Measuring Moral LLM Responses in Multilingual Capacitiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SoK: Robustness in Large Language Models against Jailbreak Attacks
  2. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks
  3. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
  4. 2025Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversion
  5. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Models
  6. 2025Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?
  7. 2026MultiBreak: A Scalable and Diverse Multi-turn Jailbreak Benchmark for Evaluating LLM Safety
  8. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models