mlsanityData to 7 Oct 2026

HateCheck: Functional Tests for Hate Speech Detection Models

arXiv 2012.15606 ↗First posted 31 Dec 2020cs.CLcited by 215

Not on the map. Why?

Cited by 22 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Detecting online hate is a difficult task that even state-of-the-art models struggle with. Typically, hate speech detection models are evaluated by measuring their performance on held-out test data using metrics such as accuracy and F1 score. However, this approach makes it difficult to identify specific model weak points. It also risks overestimating generalisable model performance due to increasingly well-evidenced systematic gaps and biases in hate speech datasets. To enable more targeted diagnostic insights, we introduce HateCheck, a suite of functional tests for hate speech detection models. We specify 29 model functionalities motivated by a review of previous research and a series of interviews with civil society stakeholders. We craft test cases for each functionality and validate their quality through a structured annotation process. To illustrate HateCheck's utility, we test near-state-of-the-art transformer models as well as two popular commercial models, revealing critical model weaknesses.

Key facts. Cited by 215 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 39; 2023: 46; 2024: 41; 2025: 47; 2026: 30.

215 papers.

Sort:

By area:

  1. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  2. 2022Red Teaming Language Models with Language Modelscited by 983
  3. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  4. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  5. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  6. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  7. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  8. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  9. 2026On-Policy Context Distillation for Language Modelscited by 161
  10. 2023Auditing large language models: a three-layered approachcited by 135
  11. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  12. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
38 more
  1. 2022Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotationcited by 67
  2. 2021Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarkingcited by 53
  3. 2022Perturbation Augmentation for Fairer NLPcited by 51
  4. 2022Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Modelscited by 51
  5. 2021Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hatecited by 50
  6. 2024Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablationcited by 48
  7. 2023Can Language Models be Instructed to Protect Personal Information?cited by 46
  8. 2022Handling and Presenting Harmful Text in NLP Researchcited by 43
  9. 2025Agentic Web: Weaving the Next Web with AI Agentscited by 43
  10. 2022Quantifying Social Biases Using Templates is Unreliablecited by 38
  11. 2023You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Contentcited by 37
  12. 2023COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statementscited by 34
  13. 2023Query-Efficient Black-Box Red Teaming via Bayesian Optimizationcited by 31
  14. 2023Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Datacited by 31
  15. 2024Whispering Experts: Neural Interventions for Toxicity Mitigation in Language Modelscited by 31
  16. 2023Assessing Language Model Deployment with Risk Cardscited by 30
  17. 2022Evaluating Psychological Safety of Large Language Modelscited by 26
  18. 2023Interpretable Unified Language Checkingcited by 26
  19. 2023From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Modelscited by 26
  20. 2024Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Modelscited by 26
  21. 2023People Make Better Edits: Measuring the Efficacy of LLM-Generated Counterfactually Augmented Data for Harmful Language Detectioncited by 25
  22. 2023KoSBi: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Applicationcited by 24
  23. 2024From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasetscited by 23
  24. 2021Multilingual Offensive Language Identification for Low-resource Languagescited by 22
  25. 2021Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysiscited by 22
  26. 2022Listening to Affected Communities to Define Extreme Speech: Dataset and Experimentscited by 21
  27. 2024NLP for Counterspeech against Hate: A Survey and How-To Guidecited by 21
  28. 2022Challenges in Applying Explainability Methods to Improve the Fairness of NLP Modelscited by 20
  29. 2024Ethos: Rectifying Language Models in Orthogonal Parameter Spacecited by 20
  30. 2022Necessity and Sufficiency for Explaining Text Classifiers: A Case Study in Hate Speech Detectioncited by 18
  31. 2022Hate Speech Criteria: A Modular Approach to Task-Specific Hate Speech Definitionscited by 18
  32. 2025System Prompt Optimization with Meta-Learningcited by 18
  33. 2022HateCheckHIn: Evaluating Hindi Hate Speech Detection Modelscited by 15
  34. 2022Flexible text generation for counterfactual fairness probingcited by 13
  35. 2024Improving Adversarial Data Collection by Supporting Annotators: Lessons from GAHD, a German Hate Speech Datasetcited by 13
  36. 2024Crowd-Calibrator: Can Annotator Disagreement Inform Calibration in Subjective Tasks?cited by 13
  37. 2025Demystifying Hateful Content: Leveraging Large Multimodal Models for Hateful Meme Detection with Explainable Decisionscited by 13
  38. 2021Multi-Task Learning with Sentiment, Emotion, and Target Detection to Recognize Hate Speech and Offensive Languagecited by 12

50 most cited of 215 All 215 (JSON)