mlsanityData to 7 Oct 2026

CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models

arXiv 2501.14940 ↗First posted 24 Jan 2025cs.CL cs.AIcited by 17

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

Aligning large language models (LLMs) with human values is essential for their safe deployment and widespread adoption. Current LLM safety benchmarks often focus solely on the refusal of individual problematic queries, which overlooks the importance of the context where the query occurs and may cause undesired refusal of queries under safe contexts that diminish user experience. Addressing this gap, we introduce CASE-Bench, a Context-Aware SafEty Benchmark that integrates context into safety assessments of LLMs. CASE-Bench assigns distinct, formally described contexts to categorized queries based on Contextual Integrity theory. Additionally, in contrast to previous studies which mainly rely on majority voting from just a few annotators, we recruited a sufficient number of annotators necessary to ensure the detection of statistically significant differences among the experimental conditions based on power analysis. Our extensive analysis using CASE-Bench on various open-source and commercial LLMs reveals a substantial and significant influence of context on human judgments (p<0.0001 from a z-test), underscoring the necessity of context in safety evaluations. We also identify notable mismatches between human judgments and LLM responses, particularly in commercial models within safe contexts.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 42 of its 62 references that are in the map. First posted on arXiv 24 Jan 2025.

Builds on

42 of 62 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Automatic Pseudo-Harmful Prompt Generation for Evaluating False Refusals in Large Language Models1 Sep
  2. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal20 Jun
  3. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Models31 May
  4. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedback19 Oct
  5. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  7. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 35

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2017Deep reinforcement learning from human preferencescited by 3,846
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  5. 2024Qwen2 Technical Reportcited by 2,076
  6. 2024Mixtral of Expertscited by 1,785
  7. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  9. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  10. 2020Aligning AI With Shared Human Valuescited by 706
  11. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  12. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
23 more
  1. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  2. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  3. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  4. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  5. 2024A StrongREJECT for Empty Jailbreakscited by 352
  6. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
  7. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 224
  8. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 218
  9. 2024Foundational Challenges in Assuring Alignment and Safety of Large Language Modelscited by 193
  10. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  11. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
  12. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 147
  13. 2020Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequencescited by 133
  14. 2023SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Modelscited by 72
  15. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuningcited by 63
  16. 2021What Would Jiminy Cricket Do? Towards Agents That Behave Morallycited by 57
  17. 2020Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbationscited by 55
  18. 2024Operationalizing Contextual Integrity in Privacy-Conscious Assistantscited by 36
  19. 2023COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statementscited by 34
  20. 2024Embedding-based classifiers can detect prompt injection attackscited by 24
  21. 2023FFT: Towards Harmlessness Evaluation and Analysis for LLMs with Factuality, Fairness, Toxicitycited by 22
  22. 2018Discovering Smart Home Internet of Things Privacy Norms Using Contextual Integritycited by 10
  23. 2024Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 10.

17 papers.

Sort:

By area:

  1. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  2. 2025MTR-DuplexBench: Towards a Comprehensive Evaluation of Multi-Round Conversations for Full-Duplex Speech Language Modelscited by 20
  3. 2025Steering Multimodal Large Language Models Decoding for Context-Aware Safetycited by 6
  4. 2026MTMCS-Bench: Evaluating Contextual Safety of Multimodal Large Language Models in Multi-Turn Dialoguescited by 4
  5. 2026Useless but Safe? Benchmarking Utility Recovery with User Intent Clarification in Multi-Turn Conversationscited by 3
  6. 2025A Red Teaming Roadmap Towards System-Level Safetycited by 2
  7. 2025OutSafe-Bench: A Benchmark for Multimodal Offensive Content Detection in Large Language Modelscited by 2
  8. 2026Blind Refusal: Language Models Refuse to Help Users Evade Unjust, Absurd, and Illegitimate Rulescited by 2
  9. 2025Beyond Benchmarks: LLM Evaluation with an Anthropomorphic and Lifecycle-oriented Roadmapcited by 1
  10. 2025Beyond Context: Large Language Models' Failure to Grasp Users' Intentcited by 1
  11. 2026RuleSafe-VL: Evaluating Rule-Conditioned Decision Reasoning in Vision-Language Content Moderationcited by 1
  12. 2026MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Modelscited by 1
5 more
  1. 2026FinRED: An Expert-Guided Benchmark Generation and Evaluation Framework for Financial LLM Red-Teamingcited by 1
  2. 2026RoTRAG: Rule of Thumb Reasoning for Conversation Harm Detection with Retrieval-Augmented Generationcited by 0
  3. 2026Transient Turn Injection: Exposing Stateless Multi-Turn Vulnerabilities in Large Language Modelscited by 0
  4. 2026StealthBench: Measuring Operational Stealth in Autonomous Offensive-Security Agentscited by 0
  5. 2026AURA-Eval: Evaluation Framework for Acting Under Risk Awareness in LLM Agent Trajectoriescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SafeLawBench: Towards Safe Alignment of Large Language Models
  2. 2026SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond
  3. 2023All Languages Matter: On the Multilingual Safety of Large Language Models
  4. 2025Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Models
  5. 2024LongSafety: Enhance Safety for Long-Context LLMs
  6. 2025Personalized Safety in LLMs: A Benchmark and A Planning-Based Agent Approach
  7. 2025Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundary
  8. 2025USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Models