mlsanityData to 14 Oct 2026

From Rogue to Safe AI: The Role of Explicit Refusals in Aligning LLMs with International Humanitarian Law

arXiv 2506.06391 ↗First posted 5 Jun 2025cs.CY cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are widely used across sectors, yet their alignment with International Humanitarian Law (IHL) is not well understood. This study evaluates eight leading LLMs on their ability to refuse prompts that explicitly violate these legal frameworks, focusing also on helpfulness - how clearly and constructively refusals are communicated. While most models rejected unlawful requests, the clarity and consistency of their responses varied. By revealing the model's rationale and referencing relevant legal or safety principles, explanatory refusals clarify the system's boundaries, reduce ambiguity, and help prevent misuse. A standardised system-level safety prompt significantly improved the quality of the explanations expressed within refusals in most models, highlighting the effectiveness of lightweight interventions. However, more complex prompts involving technical language or requests for code revealed ongoing vulnerabilities. These findings contribute to the development of safer, more transparent AI systems and propose a benchmark to evaluate the compliance of LLM with IHL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 35 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

14 of 35 references.

References · 14

  1. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  2. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  3. 2020Artificial Intelligence, Values and Alignmentcited by 448
  4. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 244
  5. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 231
  6. 2024Does Refusal Training in LLMs Generalize to the Past Tense?cited by 79
  7. 2020The Radicalization Risks of GPT-3 and Advanced Neural Language Modelscited by 76
  8. 2024Don't Say No: Jailbreaking LLM by Suppressing Refusalcited by 56
  9. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safetycited by 39
  10. 2025ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrailscited by 31
  11. 2024International Scientific Report on the Safety of Advanced AI (Interim Report)cited by 19
  12. 2025Refusal Behavior in Large Language Models: A Nonlinear Perspectivecited by 13
2 more
  1. 2024Acceptable Use Policies for Foundation Modelscited by 12
  2. 2025Beyond Benchmarks: On The False Promise of AI Regulationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal
  2. 2026Defensive Refusal Bias: How Safety Alignment Fails Cyber Defenders
  3. 2026LLM Agents in Law: Taxonomy, Applications, and Challenges
  4. 2025EU-Agent-Bench: Measuring Illegal Behavior of LLM Agents Under EU Law
  5. 2025Large Language Models' Complicit Responses to Illicit Instructions across Socio-Legal Contexts
  6. 2025Safety Compliance: Rethinking LLM Safety Reasoning through the Lens of Compliance
  7. 2026The Refusal--Compliance Tradeoff: A Large-Scale Safety Behavior Audit of Large Language Models
  8. 2026Unpredictable Safety: Domain-Dependent Compliance and the Transparency Gap in Open-Weight LLMs