mlsanityData to 7 Oct 2026

Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection

arXiv 2012.15761 ↗First posted 31 Dec 2020cs.CL cs.LGcited by 217

Not on the map. Why?

Cited by 23 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a human-and-model-in-the-loop process for dynamically generating datasets and training better performing and more robust hate detection models. We provide a new dataset of ~40,000 entries, generated and labelled by trained annotators over four rounds of dynamic data creation. It includes ~15,000 challenging perturbations and each hateful entry has fine-grained labels for the type and target of hate. Hateful entries make up 54% of the dataset, which is substantially higher than comparable datasets. We show that model performance is substantially improved using this approach. Models trained on later rounds of data collection perform better on test sets and are harder for annotators to trick. They also perform better on HateCheck, a suite of functional tests for online hate detection. We provide the code, dataset and annotation guidelines for other researchers to use. Accepted at ACL 2021.

Key facts. Cited by 217 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 28; 2023: 37; 2024: 62; 2025: 47; 2026: 28.

217 papers.

Sort:

By area:

  1. 2023StarCoder: may the source be with you!cited by 790
  2. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  3. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  4. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  5. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  6. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  7. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  8. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
  9. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  10. 2023Large Language Model Alignment: A Surveycited by 241
  11. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 180
  12. 2021Can Machines Learn Morality? The Delphi Experimentcited by 140
38 more
  1. 2023NLPositionality: Characterizing Design Biases of Datasets and Modelscited by 91
  2. 2025Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corporacited by 86
  3. 2024Curiosity-driven Red-teaming for Large Language Modelscited by 83
  4. 2021Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speechcited by 81
  5. 2021Improving Question Answering Model Robustness with Synthetic Adversarial Data Generationcited by 77
  6. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
  7. 2021Human-Adversarial Visual Question Answeringcited by 57
  8. 2022Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Listscited by 54
  9. 2024Learning diverse attacks on large language models for robust red-teaming and safety tuningcited by 54
  10. 2021Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarkingcited by 53
  11. 2022Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Modelscited by 51
  12. 2021Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hatecited by 50
  13. 2023SemEval-2023 Task 10: Explainable Detection of Online Sexismcited by 48
  14. 2024ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectorscited by 48
  15. 2023Using In-Context Learning to Improve Dialogue Safetycited by 41
  16. 2022Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Counteringcited by 40
  17. 2022Detoxifying Text with MaRCo: Controllable Revision with Experts and Anti-Expertscited by 36
  18. 2023COBRA Frames: Contextual Reasoning about Effects and Harms of Offensive Statementscited by 34
  19. 2024KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledgecited by 33
  20. 2022Using Pre-Trained Language Models for Producing Counter Narratives Against Hate Speech: a Comparative Studycited by 30
  21. 2023Assessing Language Model Deployment with Risk Cardscited by 30
  22. 2023HARE: Explainable Hate Speech Detection with Step-by-Step Reasoningcited by 30
  23. 2024LLMs for Generating and Evaluating Counterfactuals: A Comprehensive Studycited by 30
  24. 2021Does Putting a Linguist in the Loop Improve NLU Data Collection?cited by 29
  25. 2023Social Bias Probing: Fairness Benchmarking for Language Modelscited by 29
  26. 2025Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Trainingcited by 27
  27. 2022Evaluating Psychological Safety of Large Language Modelscited by 26
  28. 2024Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Modelscited by 26
  29. 2023People Make Better Edits: Measuring the Efficacy of LLM-Generated Counterfactually Augmented Data for Harmful Language Detectioncited by 25
  30. 2024From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasetscited by 23
  31. 2024Diverse and Effective Red Teaming with Auto-generated Rewards and Multi-step Reinforcement Learningcited by 23
  32. 2023How Far Can We Extract Diverse Perspectives from Large Language Models?cited by 22
  33. 2023Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Modelscited by 21
  34. 2024Personas with Attitudes: Controlling LLMs for Diverse Data Annotationcited by 21
  35. 2024An Investigation of Large Language Models for Real-World Hate Speech Detectioncited by 20
  36. 2024SAIL: Self-Improving Efficient Online Alignment of Large Language Modelscited by 20
  37. 2024NLP Systems That Can't Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helpscited by 19
  38. 2024From One to Many: Expanding the Scope of Toxicity Mitigation in Language Modelscited by 18

50 most cited of 217 All 217 (JSON)