mlsanityData to 7 Oct 2026

ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming

arXiv 2404.08676 ↗First posted 6 Apr 2024cs.CL cs.CY cs.LGcited by 96

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

When building Large Language Models (LLMs), it is paramount to bear safety in mind and protect them with guardrails. Indeed, LLMs should never generate content promoting or normalizing harmful, illegal, or unethical behavior that may contribute to harm to individuals or society. This principle applies to both normal and adversarial use. In response, we introduce ALERT, a large-scale benchmark to assess safety based on a novel fine-grained risk taxonomy. It is designed to evaluate the safety of LLMs through red teaming methodologies and consists of more than 45k instructions categorized using our novel taxonomy. By subjecting LLMs to adversarial testing scenarios, ALERT aims to identify vulnerabilities, inform improvements, and enhance the overall safety of the language models. Furthermore, the fine-grained taxonomy enables researchers to perform an in-depth evaluation that also helps one to assess the alignment with various policies. In our experiments, we extensively evaluate 10 popular open- and closed-source LLMs and demonstrate that many of them still struggle to attain reasonable levels of safety.

Key facts. Cited by 96 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 35 of its 48 references that are in the map. First posted on arXiv 6 Apr 2024.

Builds on

35 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023The Capacity for Moral Self-Correction in Large Language Models15 Feb

Other references · 34

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2020Extracting Training Data from Large Language Modelscited by 2,144
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  9. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  10. 2021Ethical and social risks of harm from Language Modelscited by 957
  11. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  12. 2024OLMo: Accelerating the Science of Language Modelscited by 599
22 more
  1. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  2. 2023The Falcon Series of Open Language Modelscited by 526
  3. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  4. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  5. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  6. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
  7. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  8. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  9. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  10. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 227
  11. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 224
  12. 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 216
  13. 2023An Overview of Catastrophic AI Riskscited by 214
  14. 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 208
  15. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 191
  16. 2023Bias Testing and Mitigation in LLM-based Code Generationcited by 53
  17. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 46
  18. 2023ChatGPT: Applications, Opportunities, and Threatscited by 28
  19. 2023An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Modelscited by 24
  20. 2024Aurora-M: Open Source Continual Pre-training for Multilingual Language and Codecited by 10
  21. 2023Ethical Considerations and Policy Implications for Large Language Models: Guiding Responsible Development and Deploymentcited by 5
  22. 2023Amplifying Limitations, Harms and Risks of Large Language Modelscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 28; 2025: 43; 2026: 25.

96 papers.

Sort:

By area:

  1. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  2. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  3. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  4. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
  5. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documentscited by 93
  6. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 84
  7. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 75
  8. 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safetycited by 71
  9. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Datacited by 67
  10. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  11. 2024ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Userscited by 32
  12. 2024Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Modelscited by 31
38 more
  1. 2025QA-LIGN: Aligning LLMs through Constitutionally Decomposed QAcited by 28
  2. 2024Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Modelscited by 27
  3. 2025SaRO: Enhancing LLM Safety through Reasoning-based Alignmentcited by 26
  4. 2025Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarkscited by 26
  5. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
  6. 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judgecited by 23
  7. 2025RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Modelscited by 23
  8. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  9. 2024BeHonest: Benchmarking Honesty in Large Language Modelscited by 20
  10. 2025Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Modelscited by 19
  11. 2026Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teamingcited by 15
  12. 2024Superficial Safety Alignment Hypothesiscited by 14
  13. 2024ChemSafetyBench: Benchmarking LLM Safety on Chemistry Domaincited by 14
  14. 2025X-Guard: Multilingual Guard Agent for Content Moderationcited by 13
  15. 2026Falcon-H1R: Pushing the Reasoning Frontiers with a Hybrid Model for Efficient Test-Time Scalingcited by 12
  16. 2025Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Modelscited by 11
  17. 2025Aligning Spoken Dialogue Models from User Interactionscited by 11
  18. 2024SafetyAnalyst: Interpretable, Transparent, and Steerable Safety Moderation for AI Behaviorcited by 10
  19. 2025Is Safety Standard Same for Everyone? User-Specific Safety Evaluation of Large Language Modelscited by 10
  20. 2024Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitationcited by 9
  21. 2025GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methodscited by 9
  22. 2025Safety Alignment Can Be Not Superficial With Explicit Safety Signalscited by 9
  23. 2024Blockchain for Large Language Model Security and Safety: A Holistic Surveycited by 8
  24. 2025ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoningcited by 8
  25. 2025Measuring Harmfulness of Computer-Using Agentscited by 8
  26. 2024SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Modelscited by 7
  27. 2025MoVa: Towards Generalizable Classification of Human Morals and Valuescited by 7
  28. 2024FRACTURED-SORRY-Bench: Framework for Revealing Attacks in Conversational Turns Undermining Refusal Efficacy and Defenses over SORRY-Bench (Automated Multi-shot Jailbreaks)cited by 6
  29. 2024Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theorycited by 6
  30. 2024SCAR: Sparse Conditioned Autoencoders for Concept Detection and Steering in LLMscited by 6
  31. 2025Code Red! On the Harmfulness of Applying Off-the-shelf Large Language Models to Programming Taskscited by 6
  32. 2024Alignment with Preference Optimization Is All You Need for LLM Safetycited by 5
  33. 2024Chat Bankman-Fried: an Exploration of LLM Alignment in Financecited by 5
  34. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Modelscited by 5
  35. 2026When Personalization Legitimizes Risks: Uncovering Safety Vulnerabilities in Personalized Dialogue Agentscited by 5
  36. 2024TorchOpera: A Compound AI System for LLM Safetycited by 4
  37. 2025Efficient Safety Retrofitting Against Jailbreaking for LLMscited by 4
  38. 2025REVEAL: Multi-turn Evaluation of Image-Input Harms for Vision LLMcited by 4

50 most cited of 96 All 96 (JSON)