mlsanityData to 14 Oct 2026

HateXScore: A Metric Suite for Evaluating Reasoning Quality in Hate Speech Explanations

arXiv 2601.13547 ↗First posted 20 Jan 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Hateful speech detection is a key component of content moderation, yet current evaluation frameworks rarely assess why a text is deemed hateful. We introduce textsfHateXScore, a four-component metric suite designed to evaluate the reasoning quality of model explanations. It assesses (i) conclusion explicitness, (ii) faithfulness and causal grounding of quoted spans, (iii) protected group identification (policy-configurable), and (iv) logical consistency among these elements. Evaluated on six diverse hate speech datasets, textsfHateXScore is intended as a diagnostic complement to reveal interpretability failures and annotation inconsistencies that are invisible to standard metrics like Accuracy or F1. Moreover, human evaluation shows strong agreement with textsfHateXScore, validating it as a practical tool for trustworthy and transparent moderation. textcolorredDisclaimer: This paper contains sensitive content that may be disturbing to some readers.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 46 references that are in the map. First posted on arXiv 20 Jan 2026.

Builds on

23 of 46 references.

References · 23

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 736
  3. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 530
  4. 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 407
  5. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 380
  6. 2019Predicting the Type and Target of Offensive Posts in Social Mediacited by 271
  7. 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 217
  8. 2020HateCheck: Functional Tests for Hate Speech Detection Modelscited by 216
  9. 2020A Survey of the State of Explainable AI for Natural Language Processingcited by 206
  10. 2020Directions in Abusive Language Training Data: Garbage In, Garbage Outcited by 140
  11. 2019A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Mediacited by 96
  12. 2017Explaining Recurrent Neural Network Predictions in Sentiment Analysiscited by 88
11 more
  1. 2023Probing LLMs for hate speech detection: strengths and vulnerabilitiescited by 52
  2. 2021Disentangling Hate in Online Memescited by 46
  3. 2023Facilitating Fine-grained Detection of Chinese Toxic Language: Hierarchical Taxonomy, Resources, and Benchmarkscited by 46
  4. 2023Evaluating GPT-3 Generated Explanations for Hateful Content Moderationcited by 46
  5. 2025Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreementcited by 14
  6. 2024SGHateCheck: Functional Tests for Detecting Hate Speech in Low-Resource Languages of Singaporecited by 10
  7. 2024Explainability and Hate Speech: Structured Explanations Make Social Media Moderators Fastercited by 7
  8. 2025Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Studycited by 7
  9. 2024Decoding Hate: Exploring Language Models' Reactions to Hate Speechcited by 6
  10. 2025Efficient Hate Speech Detection: Evaluating 38 Models from Traditional Methods to Transformerscited by 3
  11. 2024A Survey on Automatic Online Hate Speech Detection in Low-Resource Languagescited by 2

Built on by

Not cited yet.