# To Jev or Not? Evaluating the Accuracy and Efficiency of Structured Decision Models for Hate-Speech Moderation

arXiv: https://arxiv.org/abs/2610.03324 · HTML: https://mlsanity.com/paper/2610.03324/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Builds on (most cited first, 15 of 15)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection](https://mlsanity.com/paper/2012.15761/) (2020)
- [HateCheck: Functional Tests for Hate Speech Detection Models](https://mlsanity.com/paper/2012.15606/) (2020)
- [Hate Personified: Investigating the role of LLMs in content moderation](https://mlsanity.com/paper/2410.02657/) (2024)
- [HateBench: Benchmarking Hate Speech Detectors on LLM-Generated Content and Hate Campaigns](https://mlsanity.com/paper/2501.16750/) (2025)
- [STAND-Guard: A Small Task-Adaptive Content Moderation Model](https://mlsanity.com/paper/2411.05214/) (2024)
- [Evaluating Decision Models for Text Annotation in Computational Social Science](https://mlsanity.com/paper/2609.24574/) (2026)
- [DefVerify: Do Hate Speech Models Reflect Their Dataset's Definition?](https://mlsanity.com/paper/2410.15911/) (2024)
- [A Modular Taxonomy for Hate Speech Definitions and Its Impact on Zero-Shot LLM Classification Performance](https://mlsanity.com/paper/2506.18576/) (2025)
- [Untangling Hate Speech Definitions: A Semantic Componential Analysis Across Cultures and Domains](https://mlsanity.com/paper/2411.07417/) (2024)
- [Hypothesis Engineering for Zero-Shot Hate Speech Detection](https://mlsanity.com/paper/2210.00910/) (2022)
- [CoPE: A Small Language Model for Steerable and Scalable Content Labeling](https://mlsanity.com/paper/2512.18027/) (2025)
- [xList-Hate: A Checklist-Based Framework for Interpretable and Generalizable Hate Speech Detection](https://mlsanity.com/paper/2602.05874/) (2026)

## Abstract (arXiv)

The scale of online content makes hate-speech moderation challenging, while Large Language Models (LLMs) enable harmful material to be produced and adapted more easily. Moderation therefore requires efficient classifiers that can accommodate different definitions of hate speech. Recent structured decision models accept natural-language criteria and select among specified answers, raising the question of whether they can meet these requirements without task-specific training. We present HATEDECIDE, an evaluation of six decision-model configurations on four hate-speech datasets against specialized moderation, zero-shot, commercial, and supervised baselines. We examine whether supplying a dataset's definition, or decomposing it into multiple questions, improves classification, and we measure their latency and cost. We find that commercial LLMs significantly outperform all decision models on only one dataset. Supplying definitions changes up to 28% of predictions without consistently improving classification, and decomposition significantly improves performance in only 20% of the comparisons. On a diagnostic set of test cases, the best hosted decision model comes within 1.6 macro-F1 points of the best commercial LLM at approximately 97% lower inference cost. These results identify opportunities for inexpensive moderation, while showing that explicit criteria and additional questions do not reliably improve classification.