# APEACH: Attacking Pejorative Expressions with Analysis on Crowd-Generated Hate Speech Evaluation Datasets

arXiv: https://arxiv.org/abs/2202.12459 · HTML: https://mlsanity.com/paper/2202.12459/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 17 references that are in the map.
- First posted on arXiv 25 Feb 2022.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- [Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition](https://mlsanity.com/paper/2002.10361/) (2020)
- [BEEP! Korean Corpus of Online News Comments for Toxic Speech Detection](https://mlsanity.com/paper/2005.12503/) (2020)
- [Detection of Cyberbullying Incidents on the Instagram Social Network](https://mlsanity.com/paper/1503.03909/) (2015)

## Built on by (most cited first, 8 of 8)

- [KoSBi: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Application](https://mlsanity.com/paper/2305.17701/) (2023)
- [SLM as Guardian: Pioneering AI Safety with Small Language Models](https://mlsanity.com/paper/2405.19795/) (2024)
- [BAN-PL: a Novel Polish Dataset of Banned Harmful and Offensive Content from Wykop.pl web service](https://mlsanity.com/paper/2308.10592/) (2023)
- [When Crowd Meets Persona: Creating a Large-Scale Open-Domain Persona Dialogue Corpus](https://mlsanity.com/paper/2304.00350/) (2023)
- [K-HATERS: A Hate Speech Detection Corpus in Korean with Target-Specific Ratings](https://mlsanity.com/paper/2310.15439/) (2023)
- [Automatic Construction of a Korean Toxic Instruction Dataset for Ethical Tuning of Large Language Models](https://mlsanity.com/paper/2311.18215/) (2023)
- [Revealing Trends in Datasets from the 2022 ACL and EMNLP Conferences](https://mlsanity.com/paper/2404.08666/) (2024)
- [Can NLP Tackle Hate Speech in the Real World? Stakeholder-Informed Feedback and Survey on Counterspeech](https://mlsanity.com/paper/2508.04638/) (2025)

## Abstract (arXiv)

In hate speech detection, developing training and evaluation datasets across various domains is the critical issue. Whereas, major approaches crawl social media texts and hire crowd-workers to annotate the data. Following this convention often restricts the scope of pejorative expressions to a single domain lacking generalization. Sometimes domain overlap between training corpus and evaluation set overestimate the prediction performance when pretraining language models on low-data language. To alleviate these problems in Korean, we propose APEACH that asks unspecified users to generate hate speech examples followed by minimal post-labeling. We find that APEACH can collect useful datasets that are less sensitive to the lexical overlaps between the pretraining corpus and the evaluation set, thereby properly measuring the model performance.