# ToxCCIn: Toxic Content Classification with Interpretability

arXiv: https://arxiv.org/abs/2103.01328 · HTML: https://mlsanity.com/paper/2103.01328/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 1 Mar 2021.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [Why We Need New Evaluation Metrics for NLG](https://mlsanity.com/paper/1707.06875/) (2017)
- [Ex Machina: Personal Attacks Seen at Scale](https://mlsanity.com/paper/1610.08914/) (2016)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://mlsanity.com/paper/1903.08983/) (2019)
- [Deep Learning for Hate Speech Detection in Tweets](https://mlsanity.com/paper/1706.00188/) (2017)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [Fine-Grained Analysis of Propaganda in News Articles](https://mlsanity.com/paper/1910.02517/) (2019)
- [Understanding Abuse: A Typology of Abusive Language Detection Subtasks](https://mlsanity.com/paper/1705.09899/) (2017)
- [Demoting Racial Bias in Hate Speech Detection](https://mlsanity.com/paper/2005.12246/) (2020)
- [Rationale-Augmented Convolutional Neural Networks for Text Classification](https://mlsanity.com/paper/1605.04469/) (2016)
- [A Human Evaluation of AMR-to-English Generation Systems](https://mlsanity.com/paper/2004.06814/) (2020)
- [UHH-LT at SemEval-2020 Task 12: Fine-Tuning of Pre-Trained Transformer Networks for Offensive Language Detection](https://mlsanity.com/paper/2004.11493/) (2020)
- [GUIR at SemEval-2020 Task 12: Domain-Tuned Contextualized Models for Offensive Language Detection](https://mlsanity.com/paper/2007.14477/) (2020)

## Built on by (most cited first, 10 of 10)

- [Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey](https://mlsanity.com/paper/2210.07700/) (2022)
- [Towards Building a Robust Toxicity Predictor](https://mlsanity.com/paper/2404.08690/) (2024)
- [Offensive Language Identification in Low-resourced Code-mixed Dravidian languages using Pseudo-labeling](https://mlsanity.com/paper/2108.12177/) (2021)
- [SOLD: Sinhala Offensive Language Dataset](https://mlsanity.com/paper/2212.00851/) (2022)
- [A Novel Interpretability Metric for Explaining Bias in Language Models: Applications on Multilingual Models from Southeast Asia](https://mlsanity.com/paper/2410.15464/) (2024)
- [Cracking the Code: Enhancing Implicit Hate Speech Detection through Coding Classification](https://mlsanity.com/paper/2506.04693/) (2025)
- [Bias Attribution in Filipino Language Models: Extending a Bias Interpretability Metric for Application on Agglutinative Languages](https://mlsanity.com/paper/2506.07249/) (2025)
- [Combating Toxic Language: A Review of LLM-Based Strategies for Software Engineering](https://mlsanity.com/paper/2504.15439/) (2025)
- [Concept-Based Interpretability for Toxicity Detection](https://mlsanity.com/paper/2511.16689/) (2025)
- [The Unappreciated Role of Intent in Algorithmic Moderation of Social Media Content](https://mlsanity.com/paper/2405.11030/) (2024)

## Abstract (arXiv)

Despite the recent successes of transformer-based models in terms of effectiveness on a variety of tasks, their decisions often remain opaque to humans. Explanations are particularly important for tasks like offensive language or toxicity detection on social media because a manual appeal process is often in place to dispute automatically flagged content. In this work, we propose a technique to improve the interpretability of these models, based on a simple and powerful assumption: a post is at least as toxic as its most toxic span. We incorporate this assumption into transformer models by scoring a post based on the maximum toxicity of its spans and augmenting the training process to identify correct spans. We find this approach effective and can produce explanations that exceed the quality of those provided by Logistic Regression analysis (often regarded as a highly-interpretable model), according to a human study.