mlsanityData to 7 Oct 2026

Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures

arXiv 2506.07402 ↗First posted 9 Jun 2025cs.CR cs.CLcited by 3

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Large language models (LLMs) are increasingly deployed in real-world applications, raising concerns about their security. While jailbreak attacks highlight failures under overtly harmful queries, they overlook a critical risk: incorrectly answering harmless-looking inputs can be dangerous and cause real-world harm (Implicit Harm). We systematically reformulate the LLM risk landscape through a structured quadrant perspective based on output factuality and input harmlessness, uncovering an overlooked high-risk region. To investigate this gap, we propose JailFlipBench, a benchmark aims to capture implicit harm, spanning single-modal, multimodal, and factual extension scenarios with diverse evaluation metrics. We further develop initial JailFlip attack methodologies and conduct comprehensive evaluations across multiple open-source and black-box LLMs, show that implicit harm present immediate and urgent real-world risks, calling for broader LLM safety assessments and alignment beyond conventional jailbreak paradigms.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 41 of its 45 references that are in the map. First posted on arXiv 9 Jun 2025.

Builds on

41 of 45 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025LLM-Safety Evaluations Lack Robustness4 Mar
  2. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal20 Jun
  3. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep10 Jun
  4. 2024Don't Say No: Jailbreaking LLM by Suppressing Refusal25 Apr
  5. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models23 Oct
  6. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  8. 2023Are aligned neural networks adversarially aligned?26 Jun

Other references · 33

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  7. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  8. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  9. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  10. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  11. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  12. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
21 more
  1. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  2. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  3. 2024A StrongREJECT for Empty Jailbreakscited by 352
  4. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
  5. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 222
  6. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  7. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
  8. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 160
  9. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 146
  10. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  11. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  12. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 98
  13. 2024JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMscited by 87
  14. 2024Don't Listen To Me: Understanding and Exploring Jailbreak Prompts of Large Language Modelscited by 77
  15. 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 69
  16. 2024SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Typescited by 58
  17. 2024Multimodal Situational Safetycited by 54
  18. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 28
  19. 2025The Jailbreak Tax: How Useful are Your Jailbreak Outputs?cited by 26
  20. 2024Rethinking How to Evaluate Language Model Jailbreakcited by 10
  21. 2025Prefill-level Jailbreak: A Black-Box Risk Analysis of Large Language Modelscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Lifecited by 2
  2. 2025HarmTransform: Transforming Explicit Harmful Queries into Stealthy via Multi-Agent Debatecited by 1
  3. 2026How Jailbreak Attacks Inform Safety Alignment: A Defender-Centric, Shapley-Based Evaluation of Jailbreak Contributionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy
  2. 2025Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?
  3. 2025Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs
  4. 2025Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions
  5. 2025When Style Breaks Safety: Defending LLMs Against Superficial Style Alignment
  6. 2025Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers
  7. 2026Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs
  8. 2025Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?