mlsanityData to 7 Oct 2026

Attention Eclipse: Manipulating Attention to Bypass LLM Safety-Alignment

arXiv 2502.15334 ↗First posted 21 Feb 2025cs.CR cs.AI cs.LGcited by 6

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Recent research has shown that carefully crafted jailbreak inputs can induce large language models to produce harmful outputs, despite safety measures such as alignment. It is important to anticipate the range of potential Jailbreak attacks to guide effective defenses and accurate assessment of model safety. In this paper, we present a new approach for generating highly effective Jailbreak attacks that manipulate the attention of the model to selectively strengthen or weaken attention among different parts of the prompt. By harnessing attention loss, we develop more effective jailbreak attacks, that are also transferrable. The attacks amplify the success rate of existing Jailbreak algorithms including GCG, AutoDAN, and ReNeLLM, while lowering their generation cost (for example, the amplified GCG attack achieves 91.2% ASR, vs. 67.9% for the original attack on Llama2-7B/AdvBench, using less than a third of the generation time).

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 30 of its 46 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

30 of 46 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models26 Jun
  2. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
  3. 2024When "Competency" in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Complex Ciphers16 Feb
  4. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  5. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  6. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  7. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  8. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks16 Oct
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  10. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  11. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 19

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2020Longformer: The Long-Document Transformercited by 3,362
  4. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  5. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  7. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  8. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  9. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  10. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  11. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  12. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
7 more
  1. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  2. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  3. 2020Self-Attention Attribution: Interpreting Information Interactions Inside Transformercited by 124
  4. 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 82
  5. 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completioncited by 70
  6. 2024A Systematic Review of Federated Generative Modelscited by 5
  7. 2023Attention Deficit is Ordered! Fooling Deformable Vision Transformers with Collaborative Adversarial Patchescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2026AttenMIA: LLM Membership Inference Attack through Attention Signalscited by 4
  2. 2026SafeSeek: Universal Attribution of Safety Circuits in Language Modelscited by 4
  3. 2025Align to Misalign: Automatic LLM Jailbreak with Meta-Optimized LLM Judgescited by 2
  4. 2025Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractorscited by 1
  5. 2026Bypassing Prompt Injection Detectors through Evasive Injectionscited by 0
  6. 2026Attention Is Where You Attackcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AttnGCG: Enhancing Jailbreaking Attacks on LLMs with Attention Manipulation
  2. 2024Pruning for Protection: Increasing Jailbreak Resistance in Aligned LLMs Without Fine-Tuning
  3. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models
  4. 2025Involuntary Jailbreak: On Self-Prompting Attacks
  5. 2024EnJa: Ensemble Jailbreak on Large Language Models
  6. 2024Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks
  7. 2024JailPO: A Novel Black-box Jailbreak Framework via Preference Optimization against Aligned LLMs
  8. 2025AttentionDefense: Leveraging System Prompt Attention for Explainable Defense Against Novel Jailbreaks