# ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors

arXiv: https://arxiv.org/abs/2402.16444 · HTML: https://mlsanity.com/paper/2402.16444/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 40 references that are in the map.
- First posted on arXiv 26 Feb 2024.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://mlsanity.com/paper/2307.04657/) (2023)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Toxicity in ChatGPT: Analyzing Persona-assigned Language Models](https://mlsanity.com/paper/2304.05335/) (2023)
- [A Holistic Approach to Undesired Content Detection in the Real World](https://mlsanity.com/paper/2208.03274/) (2022)
- [Multi-step Jailbreaking Privacy Attacks on ChatGPT](https://mlsanity.com/paper/2304.05197/) (2023)
- [Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection](https://mlsanity.com/paper/2012.15761/) (2020)
- [A New Generation of Perspective API: Efficient Multilingual Character-level Transformers](https://mlsanity.com/paper/2202.11176/) (2022)
- [Recipes for Safety in Open-domain Chatbots](https://mlsanity.com/paper/2010.07079/) (2020)
- [CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibility](https://mlsanity.com/paper/2307.09705/) (2023)
- [COLD: A Benchmark for Chinese Offensive Language Detection](https://mlsanity.com/paper/2201.06025/) (2022)
- [On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark](https://mlsanity.com/paper/2110.08466/) (2021)
- [Ethicist: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimation](https://mlsanity.com/paper/2307.04401/) (2023)

## Built on by (most cited first, 30 of 51)

- [Agent-SafetyBench: Evaluating the Safety of LLM Agents](https://mlsanity.com/paper/2412.14470/) (2024)
- [Qwen3Guard Technical Report](https://mlsanity.com/paper/2510.14276/) (2025)
- [A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasures](https://mlsanity.com/paper/2506.19676/) (2025)
- [The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs](https://mlsanity.com/paper/2506.11094/) (2025)
- [o3-mini vs DeepSeek-R1: Which One is Safer?](https://mlsanity.com/paper/2501.18438/) (2025)
- [Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detection](https://mlsanity.com/paper/2411.01077/) (2024)
- [SafeHarness: Lifecycle-Integrated Security Architecture for LLM-based Agent Deployment](https://mlsanity.com/paper/2604.13630/) (2026)
- [Swiss Cheese Model for AI Safety: A Taxonomy and Reference Architecture for Multi-Layered Guardrails of Foundation Model Based Agents](https://mlsanity.com/paper/2408.02205/) (2024)
- [Early External Safety Testing of OpenAI's o3-mini: Insights from the Pre-Deployment Evaluation](https://mlsanity.com/paper/2501.17749/) (2025)
- [AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement](https://mlsanity.com/paper/2502.16776/) (2025)
- [OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluation](https://mlsanity.com/paper/2512.06589/) (2025)
- [PrimeGuard: Safe and Helpful LLMs through Tuning-Free Routing](https://mlsanity.com/paper/2407.16318/) (2024)
- [STATE ToxiCN: A Benchmark for Span-level Target-Aware Toxicity Extraction in Chinese Hate Speech Detection](https://mlsanity.com/paper/2501.15451/) (2025)
- [An AI System Evaluation Framework for Advancing AI Safety: Terminology, Taxonomy, Lifecycle Mapping](https://mlsanity.com/paper/2404.05388/) (2024)
- [ASTRAL: Automated Safety Testing of Large Language Models](https://mlsanity.com/paper/2501.17132/) (2025)
- [TeleAI-Safety: A comprehensive LLM jailbreaking benchmark towards attacks, defenses, and evaluations](https://mlsanity.com/paper/2512.05485/) (2025)
- [Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward](https://mlsanity.com/paper/2404.08517/) (2024)
- [Purple-teaming LLMs with Adversarial Defender Training](https://mlsanity.com/paper/2407.01850/) (2024)
- [Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Models](https://mlsanity.com/paper/2505.16446/) (2025)
- [Hiding in Plain Sight: A Steganographic Approach to Stealthy LLM Jailbreaks](https://mlsanity.com/paper/2505.16765/) (2025)
- [LM Agents May Fail to Act on Their Own Risk Knowledge](https://mlsanity.com/paper/2508.13465/) (2025)
- [Bleeding Pathways: Vanishing Discriminability in LLM Hidden States Fuels Jailbreak Attacks](https://mlsanity.com/paper/2503.11185/) (2025)
- [JailbreaksOverTime: Detecting Jailbreak Attacks Under Distribution Shift](https://mlsanity.com/paper/2504.19440/) (2025)
- [Dynamic Optimization and Safety Indicator Injection for Jailbreaking Text-to-Image Models with Multimodal Safety Filters](https://mlsanity.com/paper/2505.18979/) (2025)
- [RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaboration](https://mlsanity.com/paper/2509.25271/) (2025)
- [Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderation](https://mlsanity.com/paper/2604.06950/) (2026)
- [SAFETY-J: Evaluating Safety with Critique](https://mlsanity.com/paper/2407.17075/) (2024)
- [How Real is Your Jailbreak? Fine-grained Jailbreak Evaluation with Anchored Reference](https://mlsanity.com/paper/2601.03288/) (2026)
- [PopAlign: Population-Level Alignment for Fair Text-to-Image Generation](https://mlsanity.com/paper/2406.19668/) (2024)
- [Fine-Grained Chinese Hate Speech Understanding: Span-Level Resources, Coded Term Lexicon, and Enhanced Detection Frameworks](https://mlsanity.com/paper/2507.11292/) (2025)

## Abstract (arXiv)

The safety of Large Language Models (LLMs) has gained increasing attention in recent years, but there still lacks a comprehensive approach for detecting safety issues within LLMs' responses in an aligned, customizable and explainable manner. In this paper, we propose ShieldLM, an LLM-based safety detector, which aligns with common safety standards, supports customizable detection rules, and provides explanations for its decisions. To train ShieldLM, we compile a large bilingual dataset comprising 14,387 query-response pairs, annotating the safety of responses based on various safety standards. Through extensive experiments, we demonstrate that ShieldLM surpasses strong baselines across four test sets, showcasing remarkable customizability and explainability. Besides performing well on standard detection datasets, ShieldLM has also been shown to be effective as a safety evaluator for advanced LLMs. ShieldLM is released at urlhttps://github.com/thu-coai/ShieldLM to support accurate and explainable safety detection under various safety standards.