# HateCheck: Functional Tests for Hate Speech Detection Models

arXiv: https://arxiv.org/abs/2012.15606 · HTML: https://mlsanity.com/paper/2012.15606/

## Key facts

- Cited by 216 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2020.

## Built on by (most cited first, 30 of 216)

- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models](https://mlsanity.com/paper/2308.01263/) (2023)
- [Dynabench: Rethinking Benchmarking in NLP](https://mlsanity.com/paper/2104.14337/) (2021)
- [ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection](https://mlsanity.com/paper/2203.09509/) (2022)
- [A Holistic Approach to Undesired Content Detection in the Real World](https://mlsanity.com/paper/2208.03274/) (2022)
- [On-Policy Context Distillation for Language Models](https://mlsanity.com/paper/2602.12275/) (2026)
- [Auditing large language models: a three-layered approach](https://mlsanity.com/paper/2302.08500/) (2023)
- [Red-Teaming for Generative AI: Silver Bullet or Security Theater?](https://mlsanity.com/paper/2401.15897/) (2024)
- [Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling](https://mlsanity.com/paper/2107.03451/) (2021)
- [Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation](https://mlsanity.com/paper/2205.00501/) (2022)
- [Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking](https://mlsanity.com/paper/2106.06052/) (2021)
- [Perturbation Augmentation for Fairer NLP](https://mlsanity.com/paper/2205.12586/) (2022)
- [Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models](https://mlsanity.com/paper/2206.09917/) (2022)
- [Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hate](https://mlsanity.com/paper/2108.05921/) (2021)
- [Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation](https://mlsanity.com/paper/2410.03415/) (2024)
- [Can Language Models be Instructed to Protect Personal Information?](https://mlsanity.com/paper/2310.02224/) (2023)
- [Agentic Web: Weaving the Next Web with AI Agents](https://mlsanity.com/paper/2507.21206/) (2025)
- [Handling and Presenting Harmful Text in NLP Research](https://mlsanity.com/paper/2204.14256/) (2022)
- [Quantifying Social Biases Using Templates is Unreliable](https://mlsanity.com/paper/2210.04337/) (2022)
- [You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Content](https://mlsanity.com/paper/2308.05596/) (2023)
- [COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statements](https://mlsanity.com/paper/2306.01985/) (2023)
- [Query-Efficient Black-Box Red Teaming via Bayesian Optimization](https://mlsanity.com/paper/2305.17444/) (2023)
- [Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Data](https://mlsanity.com/paper/2310.16524/) (2023)
- [Whispering Experts: Neural Interventions for Toxicity Mitigation in Language Models](https://mlsanity.com/paper/2407.12824/) (2024)
- [Assessing Language Model Deployment with Risk Cards](https://mlsanity.com/paper/2303.18190/) (2023)
- [Evaluating Psychological Safety of Large Language Models](https://mlsanity.com/paper/2212.10529/) (2022)
- [Interpretable Unified Language Checking](https://mlsanity.com/paper/2304.03728/) (2023)

## Abstract (arXiv)

Detecting online hate is a difficult task that even state-of-the-art models struggle with. Typically, hate speech detection models are evaluated by measuring their performance on held-out test data using metrics such as accuracy and F1 score. However, this approach makes it difficult to identify specific model weak points. It also risks overestimating generalisable model performance due to increasingly well-evidenced systematic gaps and biases in hate speech datasets. To enable more targeted diagnostic insights, we introduce HateCheck, a suite of functional tests for hate speech detection models. We specify 29 model functionalities motivated by a review of previous research and a series of interviews with civil society stakeholders. We craft test cases for each functionality and validate their quality through a structured annotation process. To illustrate HateCheck's utility, we test near-state-of-the-art transformer models as well as two popular commercial models, revealing critical model weaknesses.