# Measuring a hate speech spectrum with faceted Rasch item response theory and perspective-aware, explainable-by-design deep learning

arXiv: https://arxiv.org/abs/2009.10277 · HTML: https://mlsanity.com/paper/2009.10277/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Sep 2020.

## Built on by (most cited first, 30 of 68)

- [The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation](https://mlsanity.com/paper/2211.02570/) (2022)
- [Detoxifying Language Models Risks Marginalizing Minority Voices](https://mlsanity.com/paper/2104.06390/) (2021)
- [You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Content](https://mlsanity.com/paper/2308.05596/) (2023)
- [Robust Hate Speech Detection in Social Media: A Cross-Dataset Empirical Evaluation](https://mlsanity.com/paper/2307.01680/) (2023)
- [Safer-Instruct: Aligning Language Models with Automated Preference Data](https://mlsanity.com/paper/2311.08685/) (2023)
- [From Languages to Geographies: Towards Evaluating Cultural Bias in Hate Speech Datasets](https://mlsanity.com/paper/2404.17874/) (2024)
- [How Hate Speech Varies by Target Identity: A Computational Analysis](https://mlsanity.com/paper/2210.10839/) (2022)
- [Interpretable-by-Design Text Understanding with Iteratively Generated Concept Bottleneck](https://mlsanity.com/paper/2310.19660/) (2023)
- [Understanding Counterspeech for Online Harm Mitigation](https://mlsanity.com/paper/2307.04761/) (2023)
- [Towards Interpretable Hate Speech Detection using Large Language Model-extracted Rationales](https://mlsanity.com/paper/2403.12403/) (2024)
- [MetaHate: A Dataset for Unifying Efforts on Hate Speech Detection](https://mlsanity.com/paper/2401.06526/) (2024)
- [Algorithmic Arbitrariness in Content Moderation](https://mlsanity.com/paper/2402.16979/) (2024)
- [Legilimens: Practical and Unified Content Moderation for Large Language Model Services](https://mlsanity.com/paper/2408.15488/) (2024)
- [D3CODE: Disentangling Disagreements in Data across Cultures on Offensiveness Detection and Evaluation](https://mlsanity.com/paper/2404.10857/) (2024)
- [Don't Blame the Data, Blame the Model: Understanding Noise and Bias When Learning from Subjective Annotations](https://mlsanity.com/paper/2403.04085/) (2024)
- [Utilizing Human Behavior Modeling to Manipulate Explanations in AI-Assisted Decision Making: The Good, the Bad, and the Scary](https://mlsanity.com/paper/2411.10461/) (2024)
- [Causality Guided Disentanglement for Cross-Platform Hate Speech Detection](https://mlsanity.com/paper/2308.02080/) (2023)
- [On Fairness of Low-Rank Adaptation of Large Models](https://mlsanity.com/paper/2405.17512/) (2024)
- [Political DEBATE: Efficient Zero-shot and Few-shot Classifiers for Political Text](https://mlsanity.com/paper/2409.02078/) (2024)
- [Hate Speech Detection with Generalizable Target-aware Fairness](https://mlsanity.com/paper/2406.00046/) (2024)
- [DefVerify: Do Hate Speech Models Reflect Their Dataset's Definition?](https://mlsanity.com/paper/2410.15911/) (2024)
- [PEACE: Cross-Platform Hate Speech Detection- A Causality-guided Framework](https://mlsanity.com/paper/2306.08804/) (2023)
- [The Unseen Targets of Hate -- A Systematic Review of Hateful Communication Datasets](https://mlsanity.com/paper/2405.08562/) (2024)
- [Modeling Uncertainty in Personalized Emotion Prediction with Normalizing Flows](https://mlsanity.com/paper/2312.06034/) (2023)
- [Which one is more toxic? Findings from Jigsaw Rate Severity of Toxic Comments](https://mlsanity.com/paper/2206.13284/) (2022)
- [Towards Safer AI Moderation: Evaluating LLM Moderators Through a Unified Benchmark Dataset and Advocating a Human-First Approach](https://mlsanity.com/paper/2508.07063/) (2025)
- [On Fairness of Task Arithmetic: The Role of Task Vectors](https://mlsanity.com/paper/2505.24262/) (2025)
- [Exploiting Explainability to Design Adversarial Attacks and Evaluate Attack Resilience in Hate-Speech Detection Models](https://mlsanity.com/paper/2305.18585/) (2023)
- [Towards Model-Based Data Acquisition for Subjective Multi-Task NLP Problems](https://mlsanity.com/paper/2312.08198/) (2023)
- [Can Humans Identify Domains?](https://mlsanity.com/paper/2404.01785/) (2024)

## Abstract (arXiv)

We propose a system for measuring hate speech on a continuous, interval-valued spectrum ranging from genocidal to supportive speech by combining supervised deep learning with faceted Rasch item response theory (IRT). We decompose the theoretical construct of hate speech into constituent concepts operationalized as 10 ordinal labels. Those labels are reconstituted via IRT probabilistic latent modeling into an interval outcome measure while simultaneously estimating and adjusting for each annotator's labeling perspective. Our scaling procedure integrates naturally with a multitask deep learning architecture for automated prediction, allowing design-based explainability of the continuous score through those components. We apply this method to a new, open source dataset of 50,070 social media comments sourced from YouTube, Twitter, and Reddit, annotated and labeled by 11,143 United States-based Amazon Mechanical Turk workers. Our RoBERTa-based model shows improved accuracy compared to alternative approaches. This system offers a new paradigm for supervised NLP that encourages continuous rather than binary constructs, and design-based incorporation of annotator perspective and model explainability.