mlsanityData to 7 Oct 2026

ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors

arXiv 2402.16444 ↗First posted 26 Feb 2024cs.CLcited by 48

Not on the map. Why?

Cited by 23 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The safety of Large Language Models (LLMs) has gained increasing attention in recent years, but there still lacks a comprehensive approach for detecting safety issues within LLMs' responses in an aligned, customizable and explainable manner. In this paper, we propose ShieldLM, an LLM-based safety detector, which aligns with common safety standards, supports customizable detection rules, and provides explanations for its decisions. To train ShieldLM, we compile a large bilingual dataset comprising 14,387 query-response pairs, annotating the safety of responses based on various safety standards. Through extensive experiments, we demonstrate that ShieldLM surpasses strong baselines across four test sets, showcasing remarkable customizability and explainability. Besides performing well on standard detection datasets, ShieldLM has also been shown to be effective as a safety evaluator for advanced LLMs. ShieldLM is released at urlhttps://github.com/thu-coai/ShieldLM to support accurate and explainable safety detection under various safety standards.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 19 of its 40 references that are in the map. First posted on arXiv 26 Feb 2024.

Builds on

19 of 40 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  5. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  6. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  7. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  8. 2023Baichuan 2: Open Large-scale Language Modelscited by 664
  9. 2020Recipes for building an open-domain chatbotcited by 593
  10. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  11. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  12. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
7 more
  1. 2020Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detectioncited by 217
  2. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 181
  3. 2020Recipes for Safety in Open-domain Chatbotscited by 168
  4. 2023CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibilitycited by 80
  5. 2022COLD: A Benchmark for Chinese Offensive Language Detectioncited by 74
  6. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  7. 2023Ethicist: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimationcited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 20; 2026: 19.

48 papers.

Sort:

By area:

  1. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 280
  2. 2025Qwen3Guard Technical Reportcited by 125
  3. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  4. 2025o3-mini vs DeepSeek-R1: Which One is Safer?cited by 24
  5. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
  6. 2024Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detectioncited by 20
  7. 2026SafeHarness: Lifecycle-Integrated Security Architecture for LLM-based Agent Deploymentcited by 14
  8. 2024Swiss Cheese Model for AI Safety: A Taxonomy and Reference Architecture for Multi-Layered Guardrails of Foundation Model Based Agentscited by 13
  9. 2025Early External Safety Testing of OpenAI's o3-mini: Insights from the Pre-Deployment Evaluationcited by 13
  10. 2025AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvementcited by 11
  11. 2024PrimeGuard: Safe and Helpful LLMs through Tuning-Free Routingcited by 10
  12. 2025OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluationcited by 10
36 more
  1. 2025STATE ToxiCN: A Benchmark for Span-level Target-Aware Toxicity Extraction in Chinese Hate Speech Detectioncited by 9
  2. 2024An AI System Evaluation Framework for Advancing AI Safety: Terminology, Taxonomy, Lifecycle Mappingcited by 7
  3. 2025ASTRAL: Automated Safety Testing of Large Language Modelscited by 7
  4. 2025Hiding in Plain Sight: A Steganographic Approach to Stealthy LLM Jailbreakscited by 6
  5. 2024Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forwardcited by 5
  6. 2025Bleeding Pathways: Vanishing Discriminability in LLM Hidden States Fuels Jailbreak Attackscited by 5
  7. 2025Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Modelscited by 5
  8. 2025Dynamic Optimization and Safety Indicator Injection for Jailbreaking Text-to-Image Models with Multimodal Safety Filterscited by 5
  9. 2024Purple-teaming LLMs with Adversarial Defender Trainingcited by 4
  10. 2025LM Agents May Fail to Act on Their Own Risk Knowledgecited by 4
  11. 2024SAFETY-J: Evaluating Safety with Critiquecited by 3
  12. 2025RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaborationcited by 3
  13. 2026Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderationcited by 3
  14. 2024PopAlign: Population-Level Alignment for Fair Text-to-Image Generationcited by 2
  15. 2025Fine-Grained Chinese Hate Speech Understanding: Span-Level Resources, Coded Term Lexicon, and Enhanced Detection Frameworkscited by 2
  16. 2025Libra: Large Chinese-based Safeguard for AI Contentcited by 2
  17. 2026How Real is Your Jailbreak? Fine-grained Jailbreak Evaluation with Anchored Referencecited by 2
  18. 2025Be a Multitude to Itself: A Prompt Evolution Framework for Red Teamingcited by 1
  19. 2025HauntAttack: When Attack Follows Reasoning as a Shadowcited by 1
  20. 2026Jailbreaking Large Language Models through Iterative Tool-Disguised Attacks via Reinforcement Learningcited by 1
  21. 2026MIDAS: Multi-Image Dispersion and Semantic Reconstruction for Jailbreaking MLLMscited by 1
  22. 2026Erased but Exploitable: Black-box Embedding-Aware Prompting Against Unlearned Text-to-Image Diffusion Modelscited by 1
  23. 2026SciTrace: Trajectory-Aware Safety Reasoning for Scientific Discovery Agentscited by 1
  24. 2026SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailingcited by 1
  25. 2025Gradient Co-occurrence Analysis for Detecting Unsafe Prompts in Large Language Modelscited by 0
  26. 2026Unveiling Covert Toxicity in Multimodal Data via Toxicity Association Graphs: A Graph-Based Metric and Interpretable Detection Frameworkcited by 0
  27. 2026Survive at All Costs: Exploring LLM's Risky Behaviors under Survival Pressurecited by 0
  28. 2026Evolving Jailbreaks: Automated Multi-Objective Long-Tail Attacks on Large Language Modelscited by 0
  29. 2026Exclusive Unlearningcited by 0
  30. 2026Beyond "I cannot fulfill this request": Alleviating Rigid Rejection in LLMs via Label Enhancementcited by 0
  31. 2026AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrationscited by 0
  32. 2026Data Flow Control: Data Safety Policies for AI Agentscited by 0
  33. 2026Agentic Data Environmentscited by 0
  34. 2026TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviorscited by 0
  35. 2026Combating Instruction Conflict via Energy-Driven Latent Conflict Detectioncited by 0
  36. 2026MATE: Policy-Aware Security Auditing for Mobile Agents via Synthesis-Driven Trajectory Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025From Judgment to Interference: Early Stopping LLM Harmful Outputs via Streaming Content Monitoring
  2. 2023All Languages Matter: On the Multilingual Safety of Large Language Models
  3. 2025SafeLawBench: Towards Safe Alignment of Large Language Models
  4. 2024LongSafety: Enhance Safety for Long-Context LLMs
  5. 2025LLM-Safety Evaluations Lack Robustness
  6. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment
  7. 2026Multilingual Safety Signals Are Multi-Layered: Filtering Safety-Degrading Data for Safer LLMs
  8. 2025Responsible Federated LLMs via Safety Filtering and Constitutional AI