mlsanityData to 14 Oct 2026

SafeLawBench: Towards Safe Alignment of Large Language Models

arXiv 2506.06636 ↗First posted 7 Jun 2025cs.CLcited by 9

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

With the growing prevalence of large language models (LLMs), the safety of LLMs has raised significant concerns. However, there is still a lack of definitive standards for evaluating their safety due to the subjective nature of current safety benchmarks. To address this gap, we conducted the first exploration of LLMs' safety evaluation from a legal perspective by proposing the SafeLawBench benchmark. SafeLawBench categorizes safety risks into three levels based on legal standards, providing a systematic and comprehensive framework for evaluation. It comprises 24,860 multi-choice questions and 1,106 open-domain question-answering (QA) tasks. Our evaluation included 2 closed-source LLMs and 18 open-source LLMs using zero-shot and few-shot prompting, highlighting the safety features of each model. We also evaluated the LLMs' safety-related reasoning stability and refusal behavior. Additionally, we found that a majority voting mechanism can enhance model performance. Notably, even leading SOTA models like Claude-3.5-Sonnet and GPT-4o have not exceeded 80.5% accuracy in multi-choice tasks on SafeLawBench, while the average accuracy of 20 LLMs remains at 68.8%. We urge the community to prioritize research on the safety of LLMs.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 28 of its 56 references that are in the map. First posted on arXiv 7 Jun 2025.

Builds on

28 of 56 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference20 Jun
  2. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal20 Jun
  3. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep10 Jun
  4. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedback19 Oct
  5. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset10 Jul

Other references · 23

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  5. 2023Mistral 7Bcited by 3,164
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  7. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  8. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  9. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  10. 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agentscited by 288
  11. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 223
  12. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
11 more
  1. 2024Aligner: Efficient Alignment by Learning to Correctcited by 98
  2. 2024On Protecting the Data Privacy of Large Language Models (LLMs): A Surveycited by 96
  3. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  4. 2024Are Your LLMs Capable of Stable Reasoning?cited by 65
  5. 2023Elo Uncovered: Robustness and Best Practices in Language Model Evaluationcited by 61
  6. 2024Align Anything: Training All-Modality Models to Follow Instructions with Language Feedbackcited by 39
  7. 2025Mitigating Deceptive Alignment via Self-Monitoringcited by 29
  8. 2025Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Mergingcited by 23
  9. 2024AI Governance and Accountability: An Analysis of Anthropic's Claudecited by 14
  10. 2024Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspacecited by 14
  11. 2024Dynamic Sentiment Analysis with Local Large Language Models using Majority Voting: A Study on Factors Affecting Restaurant Evaluationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 4.

9 papers.

  1. 2025Towards Advanced Mathematical Reasoning for LLMs via First-Order Logic Theorem Provingcited by 8
  2. 2025Unified Defense for Large Language Models against Jailbreak and Fine-Tuning Attacks in Educationcited by 5
  3. 2026Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verificationcited by 5
  4. 2025Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberationcited by 4
  5. 2025EU-Agent-Bench: Measuring Illegal Behavior of LLM Agents Under EU Lawcited by 1
  6. 2025HKGAI-V1: Towards Regional Sovereign Large Language Model for Hong Kongcited by 0
  7. 2026Towards Multi-modal Multi-turn Safety: From Agentic Interaction to Strategic Alignmentcited by 0
  8. 2026ARMOR 2025: A Military-Aligned Benchmark for Evaluating Large Language Model Safety Beyond Civilian Contextscited by 0
  9. 2026FinRT: Distilling Adaptive Red-Teaming Strategies into Reusable Adversarial Generators in Consumer Financecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SafeBench: A Safety Evaluation Framework for Multimodal Large Language Models
  2. 2023Fake Alignment: Are LLMs Really Aligned Well?
  3. 2025CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models
  4. 2026SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond
  5. 2023All Languages Matter: On the Multilingual Safety of Large Language Models
  6. 2026RAS: Measuring LLM Safety Through Refusal Alignment
  7. 2024S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models
  8. 2025Is Safety Standard Same for Everyone? User-Specific Safety Evaluation of Large Language Models