mlsanityData to 7 Oct 2026

XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models

arXiv 2308.01263 ↗First posted 2 Aug 2023cs.CL cs.AIcited by 477

Not on the map. Why?

Cited by 232 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Without proper safeguards, large language models will readily follow malicious instructions and generate toxic content. This risk motivates safety efforts such as red-teaming and large-scale feedback learning, which aim to make models both helpful and harmless. However, there is a tension between these two objectives, since harmlessness requires models to refuse to comply with unsafe prompts, and thus not be helpful. Recent anecdotal evidence suggests that some models may have struck a poor balance, so that even clearly safe prompts are refused if they use similar language to unsafe prompts or mention sensitive topics. In this paper, we introduce a new test suite called XSTest to identify such eXaggerated Safety behaviours in a systematic way. XSTest comprises 250 safe prompts across ten prompt types that well-calibrated models should not refuse to comply with, and 200 unsafe prompts as contrasts that models, for most applications, should refuse. We describe XSTest's creation and composition, and then use the test suite to highlight systematic failure modes in state-of-the-art language models as well as more general challenges in building safer language models.

Key facts. Cited by 477 papers in the map (data through 7 Oct 2026). Builds on 32 of its 52 references that are in the map. First posted on arXiv 2 Aug 2023.

Builds on

32 of 52 references.

References · 32

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  9. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  10. 2022Red Teaming Language Models with Language Modelscited by 983
  11. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  12. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
20 more
  1. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  2. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  3. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 668
  4. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  5. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  6. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 361
  7. 2023Pretraining Language Models with Human Preferencescited by 245
  8. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 227
  9. 2020Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detectioncited by 217
  10. 2020HateCheck: Functional Tests for Hate Speech Detection Modelscited by 215
  11. 2023On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspectivecited by 208
  12. 2021Challenges in Detoxifying Language Modelscited by 169
  13. 2020Better Fine-Tuning by Reducing Representational Collapsecited by 156
  14. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  15. 2020Contextualizing Hate Speech Classifiers with Post-hoc Explanationcited by 84
  16. 2022Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Listscited by 54
  17. 2022Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Modelscited by 51
  18. 2021Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hatecited by 50
  19. 2022SafeText: A Benchmark for Exploring Physical Safety in Language Modelscited by 41
  20. 2020The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestionscited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 74; 2025: 153; 2026: 235.

477 papers.

Sort:

By area:

  1. 2024OpenAI o1 System Cardcited by 1,220
  2. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  3. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  4. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  5. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  6. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  7. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  8. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  9. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  10. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  11. 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 294
  12. 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 292
38 more
  1. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 291
  2. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 287
  3. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  4. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  5. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
  6. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  7. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  8. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  9. 2024RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Stylecited by 168
  10. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 160
  11. 2024Safety Layers in Aligned Large Language Models: The Key to LLM Securitycited by 126
  12. 2025Qwen3Guard Technical Reportcited by 125
  13. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  14. 2025Checklists Are Better Than Reward Models For Aligning Language Modelscited by 115
  15. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
  16. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 111
  17. 2024Rule Based Rewards for Language Model Safetycited by 106
  18. 2025Large Language Models Often Know When They Are Being Evaluatedcited by 105
  19. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 104
  20. 2023Evil Geniuses: Delving into the Safety of LLM-based Agentscited by 103
  21. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  22. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  23. 2024Representation Noising: A Defence Mechanism Against Harmful Finetuningcited by 93
  24. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  25. 2025GuardReasoner: Towards Reasoning-based LLM Safeguardscited by 89
  26. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  27. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  28. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 75
  29. 2025LLMs Encode Harmfulness and Refusal Separatelycited by 75
  30. 2024Safeguarding Large Language Models: A Surveycited by 73
  31. 2023SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Modelscited by 72
  32. 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 72
  33. 2024Safety of Multimodal Large Language Models on Images and Textscited by 70
  34. 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 69
  35. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Datacited by 67
  36. 2025Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samplescited by 66
  37. 2025Effectively Controlling Reasoning Models through Thinking Interventioncited by 63
  38. 2024Building Guardrails for Large Language Modelscited by 61

50 most cited of 477 All 477 (JSON)