# Concept-Based Interpretability for Toxicity Detection

arXiv: https://arxiv.org/abs/2511.16689 · HTML: https://mlsanity.com/paper/2511.16689/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 15 Nov 2025.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [On Completeness-aware Concept-Based Explanations in Deep Neural Networks](https://mlsanity.com/paper/1910.07969/) (2019)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://mlsanity.com/paper/1903.08983/) (2019)
- [SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)](https://mlsanity.com/paper/2006.07235/) (2020)
- [GPTEval: A Survey on Assessments of ChatGPT and GPT-4](https://mlsanity.com/paper/2308.12488/) (2023)
- [Demoting Racial Bias in Hate Speech Detection](https://mlsanity.com/paper/2005.12246/) (2020)
- [Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations](https://mlsanity.com/paper/2001.05495/) (2020)
- [Rationale-Augmented Convolutional Neural Networks for Text Classification](https://mlsanity.com/paper/1605.04469/) (2016)
- [Regression Concept Vectors for Bidirectional Explanations in Histopathology](https://mlsanity.com/paper/1904.04520/) (2019)
- [Hierarchical Attention Network for Explainable Depression Detection on Twitter Aided by Metaphor Concept Mappings](https://mlsanity.com/paper/2209.07494/) (2022)
- [Concept Gradient: Concept-based Interpretation Without Linear Assumption](https://mlsanity.com/paper/2208.14966/) (2022)
- [Improving Generalizability in Implicitly Abusive Language Detection with Concept Activation Vectors](https://mlsanity.com/paper/2204.02261/) (2022)
- [ToxCCIn: Toxic Content Classification with Interpretability](https://mlsanity.com/paper/2103.01328/) (2021)
- [BERT-Based Sentiment Analysis: A Software Engineering Perspective](https://mlsanity.com/paper/2106.02581/) (2021)
- [GUIR at SemEval-2020 Task 12: Domain-Tuned Contextualized Models for Offensive Language Detection](https://mlsanity.com/paper/2007.14477/) (2020)
- [Concept-Based Explanations to Test for False Causal Relationships Learned by Abusive Language Classifiers](https://mlsanity.com/paper/2307.01900/) (2023)

## Built on by (most cited first, 1 of 1)

- [Are LLMs Ready for Conflict Monitoring? Empirical Evidence from West Africa](https://mlsanity.com/paper/2605.04177/) (2026)

## Abstract (arXiv)

The rise of social networks has not only facilitated communication but also allowed the spread of harmful content. Although significant advances have been made in detecting toxic language in textual data, the exploration of concept-based explanations in toxicity detection remains limited. In this study, we leverage various subtype attributes present in toxicity detection datasets, such as obscene, threat, insult, identity attack, and sexual explicit as concepts that serve as strong indicators to identify whether language is toxic. However, disproportionate attribution of concepts towards the target class often results in classification errors. Our work introduces an interpretability technique based on the Concept Gradient (CG) method which provides a more causal interpretation by measuring how changes in concepts directly affect the output of the model. This is an extension of traditional gradient-based methods in machine learning, which often focus solely on input features. We propose the curation of Targeted Lexicon Set, which captures toxic words that contribute to misclassifications in text classification models. To assess the significance of these lexicon sets in misclassification, we compute Word-Concept Alignment (WCA) scores, which quantify the extent to which these words lead to errors due to over-attribution to toxic concepts. Finally, we introduce a lexicon-free augmentation strategy by generating toxic samples that exclude predefined toxic lexicon sets. This approach allows us to examine whether over-attribution persists when explicit lexical overlap is removed, providing insights into the model's attribution on broader toxic language patterns.