# Challenges for Toxic Comment Classification: An In-Depth Error Analysis

arXiv: https://arxiv.org/abs/1809.07572 · HTML: https://mlsanity.com/paper/1809.07572/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2018.

## Built on by (most cited first, 30 of 56)

- [Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment](https://mlsanity.com/paper/2308.05374/) (2023)
- [Jury Learning: Integrating Dissenting Voices into Machine Learning Models](https://mlsanity.com/paper/2202.02950/) (2022)
- [Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning](https://mlsanity.com/paper/2106.02584/) (2021)
- [Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset](https://mlsanity.com/paper/2207.00220/) (2022)
- [Towards generalisable hate speech detection: a review on obstacles and solutions](https://mlsanity.com/paper/2102.08886/) (2021)
- [Multilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models](https://mlsanity.com/paper/2206.09917/) (2022)
- [Multi-lingual and Multi-cultural Figurative Language Understanding](https://mlsanity.com/paper/2305.16171/) (2023)
- [Detecting Cross-Geographic Biases in Toxicity Modeling on Social Media](https://mlsanity.com/paper/2104.06999/) (2021)
- [Unmasking and Improving Data Credibility: A Study with Datasets for Training Harmless Language Models](https://mlsanity.com/paper/2311.11202/) (2023)
- [Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editing](https://mlsanity.com/paper/2407.08770/) (2024)
- [Data Expansion using Back Translation and Paraphrasing for Hate Speech Detection](https://mlsanity.com/paper/2106.04681/) (2021)
- [NLP Systems That Can't Tell Use from Mention Censor Counterspeech, but Teaching the Distinction Helps](https://mlsanity.com/paper/2404.01651/) (2024)
- [Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimation](https://mlsanity.com/paper/2107.04212/) (2021)
- [Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements](https://mlsanity.com/paper/2302.09270/) (2023)
- [MultiParaDetox: Extending Text Detoxification with Parallel Data to New Languages](https://mlsanity.com/paper/2404.02037/) (2024)
- [Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation](https://mlsanity.com/paper/2312.01648/) (2023)
- [Deciphering Implicit Hate: Evaluating Automated Detection Algorithms for Multimodal Hate](https://mlsanity.com/paper/2106.05903/) (2021)
- [Automated Identification of Toxic Code Reviews Using ToxiCR](https://mlsanity.com/paper/2202.13056/) (2022)
- [Harmful YouTube Video Detection: A Taxonomy of Online Harm and MLLMs as Alternative Annotators](https://mlsanity.com/paper/2411.05854/) (2024)
- [From LLM-anation to LLM-orchestrator: Coordinating Small Models for Data Labeling](https://mlsanity.com/paper/2506.16393/) (2025)
- [Large Language Models can be Strong Self-Detoxifiers](https://mlsanity.com/paper/2410.03818/) (2024)
- [DefVerify: Do Hate Speech Models Reflect Their Dataset's Definition?](https://mlsanity.com/paper/2410.15911/) (2024)
- [Hidden behind the obvious: misleading keywords and implicitly abusive language on social media](https://mlsanity.com/paper/2205.01374/) (2022)
- [Predicting Different Types of Subtle Toxicity in Unhealthy Online Conversations](https://mlsanity.com/paper/2106.03952/) (2021)
- [Limitations of machine learning for building energy prediction: ASHRAE Great Energy Predictor III Kaggle competition error analysis](https://mlsanity.com/paper/2106.13475/) (2021)
- [From Theories on Styles to their Transfer in Text: Bridging the Gap with a Hierarchical Survey](https://mlsanity.com/paper/2110.15871/) (2021)
- [Leveraging a New Spanish Corpus for Multilingual and Crosslingual Metaphor Detection](https://mlsanity.com/paper/2210.10358/) (2022)
- [ToxBuster: In-game Chat Toxicity Buster with BERT](https://mlsanity.com/paper/2305.12542/) (2023)
- [Adding guardrails to advanced chatbots](https://mlsanity.com/paper/2306.07500/) (2023)
- [Characterizing Information Seeking Events in Health-Related Social Discourse](https://mlsanity.com/paper/2308.09156/) (2023)

## Abstract (arXiv)

Toxic comment classification has become an active research field with many recently proposed approaches. However, while these approaches address some of the task's challenges others still remain unsolved and directions for further research are needed. To this end, we compare different deep learning and shallow approaches on a new, large comment dataset and propose an ensemble that outperforms all individual models. Further, we validate our findings on a second dataset. The results of the ensemble enable us to perform an extensive error analysis, which reveals open challenges for state-of-the-art methods and directions towards pending future research. These challenges include missing paradigmatic context and inconsistent dataset labels.