# Towards Efficient and Explainable Hate Speech Detection via Model Distillation

arXiv: https://arxiv.org/abs/2412.13698 · HTML: https://mlsanity.com/paper/2412.13698/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 18 Dec 2024.

## Builds on (most cited first, 30 of 30)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [Teaching Small Language Models to Reason](https://mlsanity.com/paper/2212.08410/) (2022)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [HateBERT: Retraining BERT for Abusive Language Detection in English](https://mlsanity.com/paper/2010.12472/) (2020)
- [Hate Speech Dataset from a White Supremacy Forum](https://mlsanity.com/paper/1809.04444/) (2018)
- [A New Generation of Perspective API: Efficient Multilingual Character-level Transformers](https://mlsanity.com/paper/2202.11176/) (2022)
- [Hate Lingo: A Target-based Linguistic Analysis of Hate Speech in Social Media](https://mlsanity.com/paper/1804.04257/) (2018)
- [PINTO: Faithful Language Reasoning Using Prompt-Generated Rationales](https://mlsanity.com/paper/2211.01562/) (2022)
- [Towards Explainable NLP: A Generative Explanation Framework for Text Classification](https://mlsanity.com/paper/1811.00196/) (2018)
- [Analyzing the Targets of Hate in Online Social Media](https://mlsanity.com/paper/1603.07709/) (2016)
- ["Call me sexist, but...": Revisiting Sexism Detection Using Psychological Scales and Adversarial Samples](https://mlsanity.com/paper/2004.12764/) (2020)
- [HateMM: A Multi-Modal Dataset for Hate Video Classification](https://mlsanity.com/paper/2305.03915/) (2023)
- [Toxicity Detection with Generative Prompt-based Inference](https://mlsanity.com/paper/2205.12390/) (2022)
- [HARE: Explainable Hate Speech Detection with Step-by-Step Reasoning](https://mlsanity.com/paper/2311.00321/) (2023)
- [Peer to Peer Hate: Hate Speech Instigators and Their Targets](https://mlsanity.com/paper/1804.04649/) (2018)
- [Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Models](https://mlsanity.com/paper/2312.08303/) (2023)
- [Towards Interpretable Hate Speech Detection using Large Language Model-extracted Rationales](https://mlsanity.com/paper/2403.12403/) (2024)
- [MetaHate: A Dataset for Unifying Efforts on Hate Speech Detection](https://mlsanity.com/paper/2401.06526/) (2024)
- [Auditing Elon Musk's Impact on Hate Speech and Bots](https://mlsanity.com/paper/2304.04129/) (2023)
- [Distiller: A Systematic Study of Model Distillation Methods in Natural Language Processing](https://mlsanity.com/paper/2109.11105/) (2021)

## Built on by (most cited first, 5 of 5)

- [LLM-Based Multi-Task Bangla Hate Speech Detection: Type, Severity, and Target](https://mlsanity.com/paper/2510.01995/) (2025)
- [Interpreting LLM-as-a-Judge Policies via Verifiable Global Explanations](https://mlsanity.com/paper/2510.08120/) (2025)
- ["Harmless to You, Hurtful to Me!": Investigating the Detection of Toxic Languages Grounded in the Perspective of Youth](https://mlsanity.com/paper/2508.02094/) (2025)
- [WATCHED: A Web AI Agent Tool for Combating Hate Speech by Expanding Data](https://mlsanity.com/paper/2509.01379/) (2025)
- [Explain the Flag: Contextualizing Hate Speech Beyond Censorship](https://mlsanity.com/paper/2604.14970/) (2026)

## Abstract (arXiv)

Automatic detection of hate and abusive language is essential to combat its online spread. Moreover, recognising and explaining hate speech serves to educate people about its negative effects. However, most current detection models operate as black boxes, lacking interpretability and explainability. In this context, Large Language Models (LLMs) have proven effective for hate speech detection and to promote interpretability. Nevertheless, they are computationally costly to run. In this work, we propose distilling big language models by using Chain-of-Thought to extract explanations that support the hate speech classification task. Having small language models for these tasks will contribute to their use in operational settings. In this paper, we demonstrate that distilled models deliver explanations of the same quality as larger models while surpassing them in classification performance. This dual capability, classifying and explaining, advances hate speech detection making it more affordable, understandable and actionable.