# BEEP! Korean Corpus of Online News Comments for Toxic Speech Detection

arXiv: https://arxiv.org/abs/2005.12503 · HTML: https://mlsanity.com/paper/2005.12503/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 May 2020.

## Built on by (most cited first, 30 of 39)

- [KLUE: Korean Language Understanding Evaluation](https://mlsanity.com/paper/2105.09680/) (2021)
- [KMMLU: Measuring Massive Multitask Language Understanding in Korean](https://mlsanity.com/paper/2402.11548/) (2024)
- [Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey](https://mlsanity.com/paper/2210.07700/) (2022)
- [CLIcK: A Benchmark Dataset of Cultural and Linguistic Intelligence in Korean](https://mlsanity.com/paper/2403.06412/) (2024)
- [CultureLLM: Incorporating Cultural Differences into Large Language Models](https://mlsanity.com/paper/2402.10946/) (2024)
- [RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?](https://mlsanity.com/paper/2404.14397/) (2024)
- [A systematic review of Hate Speech automatic detection using Natural Language Processing](https://mlsanity.com/paper/2106.00742/) (2021)
- [KoSBi: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Application](https://mlsanity.com/paper/2305.17701/) (2023)
- [CulturePark: Boosting Cross-cultural Understanding in Large Language Models](https://mlsanity.com/paper/2405.15145/) (2024)
- [Exploring Methods for Cross-lingual Text Style Transfer: The Case of Text Detoxification](https://mlsanity.com/paper/2311.13937/) (2023)
- [KOLD: Korean Offensive Language Dataset](https://mlsanity.com/paper/2205.11315/) (2022)
- [Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean](https://mlsanity.com/paper/2403.10882/) (2024)
- [K-MHaS: A Multi-label Hate Speech Detection Dataset in Korean Online News Comment](https://mlsanity.com/paper/2208.10684/) (2022)
- [DEPTWEET: A Typology for Social Media Texts to Detect Depression Severities](https://mlsanity.com/paper/2210.05372/) (2022)
- [STAND-Guard: A Small Task-Adaptive Content Moderation Model](https://mlsanity.com/paper/2411.05214/) (2024)
- [SGHateCheck: Functional Tests for Detecting Hate Speech in Low-Resource Languages of Singapore](https://mlsanity.com/paper/2405.01842/) (2024)
- [APEACH: Attacking Pejorative Expressions with Analysis on Crowd-Generated Hate Speech Evaluation Datasets](https://mlsanity.com/paper/2202.12459/) (2022)
- [SLM as Guardian: Pioneering AI Safety with Small Language Models](https://mlsanity.com/paper/2405.19795/) (2024)
- [Korean Online Hate Speech Dataset for Multilabel Classification: How Can Social Science Improve Dataset on Hate Speech?](https://mlsanity.com/paper/2204.03262/) (2022)
- [When Crowd Meets Persona: Creating a Large-Scale Open-Domain Persona Dialogue Corpus](https://mlsanity.com/paper/2304.00350/) (2023)
- [K-HATERS: A Hate Speech Detection Corpus in Korean with Target-Specific Ratings](https://mlsanity.com/paper/2310.15439/) (2023)
- [A Survey on Awesome Korean NLP Datasets](https://mlsanity.com/paper/2112.01624/) (2021)
- [KoMultiText: Large-Scale Korean Text Dataset for Classifying Biased Speech in Real-World Online Services](https://mlsanity.com/paper/2310.04313/) (2023)
- [The Skipped Beat: A Study of Sociopragmatic Understanding in LLMs for 64 Languages](https://mlsanity.com/paper/2310.14557/) (2023)
- [SOLD: Sinhala Offensive Language Dataset](https://mlsanity.com/paper/2212.00851/) (2022)
- [Improving Korean NLP Tasks with Linguistically Informed Subword Tokenization and Sub-character Decomposition](https://mlsanity.com/paper/2311.03928/) (2023)
- [Don't be a Fool: Pooling Strategies in Offensive Language Detection from User-Intended Adversarial Attacks](https://mlsanity.com/paper/2403.15467/) (2024)
- [K/DA: Automated Data Generation Pipeline for Detoxifying Implicitly Offensive Language in Korean](https://mlsanity.com/paper/2506.13513/) (2025)
- [UIT-E10dot3 at SemEval-2021 Task 5: Toxic Spans Detection with Named Entity Recognition and Question-Answering Approaches](https://mlsanity.com/paper/2104.07376/) (2021)
- [Transformer-based Korean Pretrained Language Models: A Survey on Three Years of Progress](https://mlsanity.com/paper/2112.03014/) (2021)

## Abstract (arXiv)

Toxic comments in online platforms are an unavoidable social issue under the cloak of anonymity. Hate speech detection has been actively done for languages such as English, German, or Italian, where manually labeled corpus has been released. In this work, we first present 9.4K manually labeled entertainment news comments for identifying Korean toxic speech, collected from a widely used online news platform in Korea. The comments are annotated regarding social bias and hate speech since both aspects are correlated. The inter-annotator agreement Krippendorff's alpha score is 0.492 and 0.496, respectively. We provide benchmarks using CharCNN, BiLSTM, and BERT, where BERT achieves the highest score on all tasks. The models generally display better performance on bias identification, since the hate speech detection is a more subjective issue. Additionally, when BERT is trained with bias label for hate speech detection, the prediction score increases, implying that bias and hate are intertwined. We make our dataset publicly available and open competitions with the corpus and benchmarks.