mlsanityData to 14 Oct 2026

Semantic Representation Attack against Aligned Large Language Models

arXiv 2509.19360 ↗First posted 18 Sep 2025cs.CL cs.AIcited by 3

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) increasingly employ alignment techniques to prevent harmful outputs. Despite these safeguards, attackers can circumvent them by crafting prompts that induce LLMs to generate harmful content. Current methods typically target exact affirmative responses, such as ``Sure, here is...'', suffering from limited convergence, unnatural prompts, and high computational costs. We introduce Semantic Representation Attack, a novel paradigm that fundamentally reconceptualizes adversarial objectives against aligned LLMs. Rather than targeting exact textual patterns, our approach exploits the semantic representation space comprising diverse responses with equivalent harmful meanings. This innovation resolves the inherent trade-off between attack efficacy and prompt naturalness that plagues existing methods. The Semantic Representation Heuristic Search algorithm is proposed to efficiently generate semantically coherent and concise adversarial prompts by maintaining interpretability during incremental expansion. We establish rigorous theoretical guarantees for semantic convergence and demonstrate that our method achieves unprecedented attack success rates (89.41% averaged across 18 LLMs, including 100% on 11 models) while maintaining stealthiness and efficiency. Comprehensive experimental results confirm the overall superiority of our Semantic Representation Attack. The code will be publicly available.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 54 of its 77 references that are in the map. First posted on arXiv 18 Sep 2025.

Builds on

54 of 77 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Attacking Large Language Models with Projected Gradient Descent14 Feb
  2. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models23 Oct
  3. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  5. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul
  6. 2023Are aligned neural networks adversarially aligned?26 Jun

Other references · 48

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  7. 2013Intriguing properties of neural networkscited by 5,155
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  9. 2023Qwen Technical Reportcited by 3,344
  10. 2023Mistral 7Bcited by 3,164
  11. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  12. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
36 more
  1. 2016Adversarial examples in the physical worldcited by 1,769
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  3. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  4. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  5. 2022Red Teaming Language Models with Language Modelscited by 1,026
  6. 2017Ensemble Adversarial Training: Attacks and Defensescited by 840
  7. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  8. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  9. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  10. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  11. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  12. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
  13. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
  14. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  15. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  16. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
  17. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 336
  18. 2017Defense against Adversarial Attacks Using High-Level Representation Guided Denoisercited by 329
  19. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 305
  20. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 290
  21. 2019Fooling automated surveillance cameras: adversarial patches to attack person detectioncited by 268
  22. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 252
  23. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 234
  24. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
  25. 2023SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scalingcited by 172
  26. 2023Orca 2: Teaching Small Language Models How to Reasoncited by 159
  27. 2023Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Modelscited by 138
  28. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 134
  29. 2023Language Models Meet World Models: Embodied Experiences Enhance Language Modelscited by 124
  30. 2024Fast Adversarial Attacks on Language Models In One GPU Minutecited by 88
  31. 2023Large Language Models as Generalizable Policies for Embodied Taskscited by 86
  32. 2024Does Refusal Training in LLMs Generalize to the Past Tense?cited by 79
  33. 2020Semantic Equivalent Adversarial Data Augmentation for Visual Question Answeringcited by 26
  34. 2023Physical Adversarial Attacks for Surveillance: A Surveycited by 15
  35. 2024Does Safety Training of LLMs Generalize to Semantically Related Natural Prompts?cited by 15
  36. 2023Uncertainty-weighted Loss Functions for Improved Adversarial Attacks on Semantic Segmentationcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Auditing Cascading Risks in Multi-Agent Systems via Semantic-Geometric Co-evolutioncited by 1
  2. 2026LLM-Agnostic Semantic Representation Attackcited by 0
  3. 2026Eliciting Intrinsic Hallucinations in LLMs via Semantically Equivalent Adversarial Attackscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models
  2. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Models
  3. 2025REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective
  4. 2025Attack and defense techniques in large language models: A survey and new perspectives
  5. 2026One Attack to Fool Them All: Highly Transferable Black-Box Adversarial Attacks on Frontier MLLMs
  6. 2025Embedding Poisoning: Bypassing Safety Alignment via Embedding Semantic Shift
  7. 2026Reasoning Hijacking: The Fragility of Reasoning Alignment in Large Language Models
  8. 2024Can Reinforcement Learning Unlock the Hidden Dangers in Aligned Large Language Models?