mlsanityData to 14 Oct 2026

The Structural Safety Generalization Problem

arXiv 2504.09712 ↗First posted 13 Apr 2025cs.CR cs.AI cs.CVcited by 2

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

LLM jailbreaks are a widespread safety challenge. Given this problem has not yet been tractable, we suggest targeting a key failure mechanism: the failure of safety to generalize across semantically equivalent inputs. We further focus the target by requiring desirable tractability properties of attacks to study: explainability, transferability between models, and transferability between goals. We perform red-teaming within this framework by uncovering new vulnerabilities to multi-turn, multi-image, and translation-based attacks. These attacks are semantically equivalent by our design to their single-turn, single-image, or untranslated counterparts, enabling systematic comparisons; we show that the different structures yield different safety outcomes. We then demonstrate the potential for this framework to enable new defenses by proposing a Structure Rewriting Guardrail, which converts an input to a structure more conducive to safety assessment. This guardrail significantly improves refusal of harmful inputs, without over-refusing benign ones. Thus, by framing this intermediate challenge - more tractable than universal defenses but essential for long-term safety - we highlight a critical milestone for AI safety research.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 42 of its 72 references that are in the map. First posted on arXiv 13 Apr 2025.

Builds on

42 of 72 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails15 Jan
  2. 2024Does Refusal Training in LLMs Generalize to the Past Tense?16 Jul
  3. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs26 Jun
  4. 2024Refusal in Language Models Is Mediated by a Single Direction17 Jun
  5. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety8 Apr

Other references · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  6. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  7. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  8. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  9. 2016Universal adversarial perturbationscited by 784
  10. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  11. 2023Detecting Language Model Attacks with Perplexitycited by 431
  12. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
25 more
  1. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 398
  2. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
  3. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  4. 2023Low-Resource Languages Jailbreak GPT-4cited by 309
  5. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 302
  6. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 278
  7. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 216
  8. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
  9. 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 181
  10. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 151
  11. 2024Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversationscited by 129
  12. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 125
  13. 2023Don't Trust ChatGPT when Your Question is not in English: A Study of Multilingual Abilities and Types of LLMscited by 104
  14. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 90
  15. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinkingcited by 81
  16. 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 77
  17. 2023Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competitioncited by 68
  18. 2023The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensivenesscited by 67
  19. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 32
  20. 2022Adversarial Policies Beat Superhuman Go AIscited by 29
  21. 2024Aurora-M: Open Source Continual Pre-training for Multilingual Language and Codecited by 10
  22. 2024Efficient Prompt Caching via Embedding Similaritycited by 9
  23. 2021Performance Evaluation of Adversarial Attacks: Discrepancies and Solutionscited by 7
  24. 2024Explainable and Transferable Adversarial Attack for ML-Based Network Intrusion Detectorscited by 7
  25. 2021Explainable Adversarial Attacks in Deep Neural Networks Using Activation Profilescited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Ranking Free RAG: Replacing Re-ranking with Selection in RAG for Sensitive Domainscited by 5
  2. 2026kNNGuard: Turning LLM Hidden Activations into a Training-Free Configurable Guardrailcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025GraphAttack: Exploiting Representational Blindspots in LLM Safety Mechanisms
  2. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications
  3. 2026Compositional Jailbreaking: An Empirical Analysis of Mutator Chain Interactions in Aligned LLMs
  4. 2026Safety-Potential Pruning for Enhancing Safety Prompts Against VLM Jailbreaking Without Retraining
  5. 2026When Skills Meet Safety: Benchmarking and Characterizing the Adaptive Jailbreak Robustness of Skill-Merged LLMs
  6. 2025Why Safeguarded Ships Run Aground? Aligned Large Language Models' Safety Mechanisms Tend to Be Anchored in The Template Region
  7. 2026Knowing without Acting: The Disentangled Geometry of Safety Mechanisms in Large Language Models
  8. 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasets