mlsanityData to 7 Oct 2026

Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hate

arXiv 2108.05921 ↗First posted 12 Aug 2021cs.CL cs.CYcited by 50

Not on the map. Why?

Abstract (arXiv)

Detecting online hate is a complex task, and low-performing models have harmful consequences when used for sensitive applications such as content moderation. Emoji-based hate is an emerging challenge for automated detection. We present HatemojiCheck, a test suite of 3,930 short-form statements that allows us to evaluate performance on hateful language expressed with emoji. Using the test suite, we expose weaknesses in existing hate detection models. To address these weaknesses, we create the HatemojiBuild dataset using a human-and-model-in-the-loop approach. Models built with these 5,912 adversarial examples perform substantially better at detecting emoji-based hate, while retaining strong performance on text-only hate. Both HatemojiCheck and HatemojiBuild are made publicly available. See our Github Repository (https://github.com/HannahKirk/Hatemoji). HatemojiCheck, HatemojiBuild, and the final Hatemoji Model are also available on HuggingFace (https://huggingface.co/datasets/HannahRoseKirk/).

Key facts. Cited by 50 papers in the map (data through 7 Oct 2026). Builds on 25 of its 60 references that are in the map. First posted on arXiv 12 Aug 2021.

Builds on

25 of 60 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  4. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  5. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  6. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  7. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  8. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  9. 2018A Reductions Approach to Fair Classificationcited by 585
  10. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  11. 2020BERTweet: A pre-trained language model for English Tweetscited by 346
  12. 2018Large Scale Crowdsourcing and Characterization of Twitter Abusive Behaviorcited by 265
13 more
  1. 2018A Simple Method for Commonsense Reasoningcited by 223
  2. 2020Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detectioncited by 217
  3. 2020HateCheck: Functional Tests for Hate Speech Detection Modelscited by 215
  4. 2019SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)cited by 206
  5. 2019Fair Regression: Quantitative Definitions and Reduction-based Algorithmscited by 168
  6. 2020Beat the AI: Investigating Adversarial Human Annotation for Reading Comprehensioncited by 131
  7. 2017Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasmcited by 117
  8. 2017Understanding Abuse: A Typology of Abusive Language Detection Subtaskscited by 115
  9. 2020DynaSent: A Dynamic Benchmark for Sentiment Analysiscited by 63
  10. 2021Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarkingcited by 53
  11. 2016Analyzing the Targets of Hate in Online Social Mediacited by 48
  12. 2020The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestionscited by 22
  13. 2019Attending the Emotions to Detect Online Abusive Languagecited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 8; 2023: 15; 2024: 10; 2025: 13; 2026: 3.

50 papers.

Sort:

By area:

  1. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  2. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  3. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 181
  4. 2023Auditing large language models: a three-layered approachcited by 135
  5. 2025Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corporacited by 86
  6. 2024UnsafeBench: Benchmarking Image Safety Classifiers on Real-World and AI-Generated Imagescited by 52
  7. 2022Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Modelscited by 51
  8. 2023SemEval-2023 Task 10: Explainable Detection of Online Sexismcited by 48
  9. 2022Handling and Presenting Harmful Text in NLP Researchcited by 43
  10. 2023Analyzing Dataset Annotation Quality Management in the Wildcited by 39
  11. 2023Assessing Language Model Deployment with Risk Cardscited by 30
  12. 2024From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasetscited by 23
38 more
  1. 2023MTTM: Metamorphic Testing for Textual Content Moderation Softwarecited by 20
  2. 2022Hate Speech Criteria: A Modular Approach to Task-Specific Hate Speech Definitionscited by 18
  3. 2022Dynatask: A Framework for Creating Dynamic AI Benchmark Taskscited by 16
  4. 2022Flexible text generation for counterfactual fairness probingcited by 13
  5. 2024Improving Adversarial Data Collection by Supporting Annotators: Lessons from GAHD, a German Hate Speech Datasetcited by 13
  6. 2024STAND-Guard: A Small Task-Adaptive Content Moderation Modelcited by 11
  7. 2024Autoregressive Pre-Training on Pixels and Textscited by 10
  8. 2025HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate Campaignscited by 10
  9. 2023Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Modelscited by 7
  10. 2024Mitigating Exaggerated Safety in Large Language Modelscited by 7
  11. 2023Evaluating ChatGPT's Performance for Multilingual and Emoji-based Hate Speech Detectioncited by 6
  12. 2023EmojiLM: Modeling the New Emoji Languagecited by 6
  13. 2023Validating Multimedia Content Moderation Software via Semantic Fusioncited by 5
  14. 2023Break it, Imitate it, Fix it: Robustness by Generating Human-Like Attackscited by 5
  15. 2023An Image is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Softwarecited by 3
  16. 2024GPT-HateCheck: Can LLMs Write Better Functional Tests for Hate Speech Detection?cited by 3
  17. 2024BEADs: Bias Evaluation Across Domainscited by 3
  18. 2025Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directionscited by 3
  19. 2023Cross-functional Analysis of Generalisation in Behavioural Learningcited by 2
  20. 2024Specification Overfitting in Artificial Intelligencecited by 2
  21. 2025Fine-Grained Chinese Hate Speech Understanding: Span-Level Resources, Coded Term Lexicon, and Enhanced Detection Frameworkscited by 2
  22. 2025Provably Safe Model Updatescited by 2
  23. 2025Aetheria: A multimodal interpretable content safety framework based on multi-agent debate and collaborationcited by 2
  24. 2021Combining Data-driven Supervision with Human-in-the-loop Feedback for Entity Resolutioncited by 1
  25. 2025Breaking the Cloak! Unveiling Chinese Cloaked Toxicity with Homophone Graph and Toxic Lexiconcited by 1
  26. 2026Bye Bye Perspective API: Lessons for Building and Governing Measurement Infrastructurecited by 1
  27. 2022TestAug: A Framework for Augmenting Capability-based NLP Testscited by 0
  28. 2023DoDo Learning: DOmain-DemOgraphic Transfer in Language Models for Detecting Abuse Targeted at Public Figurescited by 0
  29. 2023Functionality learning through specification instructionscited by 0
  30. 2024Into the crossfire: evaluating the use of a language model to crowdsource gun violence reportscited by 0
  31. 2025Covering Cracks in Content Moderation: Delexicalized Distant Supervision for Illicit Drug Jargon Detectioncited by 0
  32. 2025Towards a comprehensive taxonomy of online abusive language informed by machine leaningcited by 0
  33. 2025The Hidden Language of Harm: Examining the Role of Emojis in Harmful Online Communication and Content Moderationcited by 0
  34. 2025Web(er) of Hate: A Survey on How Hate Speech Is Typedcited by 0
  35. 2025From Seed to Harvest: Augmenting Human Creativity with AI for Red-teaming Text-to-Image Modelscited by 0
  36. 2025Metamorphic Testing for Audio Content Moderation Softwarecited by 0
  37. 2026ContiGuard: A Framework for Continual Toxicity Detection Against Evolving Evasive Perturbationscited by 0
  38. 2026RiskChainBench: A Benchmark for Obfuscated Platform Message Restoration and Evidence-Grounded Web Investigationcited by 0