# MFTCXplain: A Multilingual Benchmark Dataset for Evaluating the Moral Reasoning of LLMs through Multi-hop Hate Speech Explanation

arXiv: https://arxiv.org/abs/2506.19073 · HTML: https://mlsanity.com/paper/2506.19073/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 12 of its 95 references that are in the map.
- First posted on arXiv 23 Jun 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 12 of 12)

- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://mlsanity.com/paper/1911.03429/) (2019)
- [Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Models](https://mlsanity.com/paper/2404.18796/) (2024)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [On Releasing Annotator-Level Labels and Information in Datasets](https://mlsanity.com/paper/2110.05699/) (2021)
- [Moral Foundations of Large Language Models](https://mlsanity.com/paper/2310.15337/) (2023)
- [Values in the Wild: Discovering and Analyzing Values in Real-World Language Model Interactions](https://mlsanity.com/paper/2504.15236/) (2025)
- [Hate Towards the Political Opponent: A Twitter Corpus Study of the 2020 US Elections on the Basis of Offensive Speech and Stance Detection](https://mlsanity.com/paper/2103.01664/) (2021)
- [Rethinking Machine Ethics -- Can LLMs Perform Moral Reasoning through the Lens of Moral Theories?](https://mlsanity.com/paper/2308.15399/) (2023)
- [The Moral Foundations Reddit Corpus](https://mlsanity.com/paper/2208.05545/) (2022)
- [Mind the Language Gap: Automated and Augmented Evaluation of Bias in LLMs for High- and Low-Resource Languages](https://mlsanity.com/paper/2504.18560/) (2025)
- [Enhancing Large Language Models with Neurosymbolic Reasoning for Multilingual Tasks](https://mlsanity.com/paper/2506.02483/) (2025)
- [DADIT: A Dataset for Demographic Classification of Italian Twitter Users and a Comparison of Prediction Methods](https://mlsanity.com/paper/2403.05700/) (2024)

## Built on by (most cited first, 8 of 8)

- [Tracing Moral Foundations in Large Language Models](https://mlsanity.com/paper/2601.05437/) (2026)
- [Self-Explaining Hate Speech Detection with Moral Rationales](https://mlsanity.com/paper/2601.03481/) (2026)
- [NormAct: Benchmarking Embodied Agents' Proactive Compliance with Unspoken Social Norms](https://mlsanity.com/paper/2606.27826/) (2026)
- [Event-Centric Human Value Understanding in News-Domain Texts: An Actor-Conditioned, Multi-Granularity Benchmark](https://mlsanity.com/paper/2603.17838/) (2026)
- [Less Is Moral: A CHARMing Framework for Moral Foundations Detection in Endorsement Behaviour](https://mlsanity.com/paper/2609.03330/) (2026)
- [CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systems](https://mlsanity.com/paper/2609.22494/) (2026)
- [FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing](https://mlsanity.com/paper/2609.25298/) (2026)
- [From Triage to Impact: A Multi-Hop Reasoning Benchmark for Explainable Persian Hate Speech](https://mlsanity.com/paper/2610.20370/) (2026)

## Abstract (arXiv)

Ensuring the moral reasoning capabilities of Large Language Models (LLMs) is a growing concern as these systems are used in socially sensitive tasks. Nevertheless, current evaluation benchmarks present two major shortcomings: a lack of annotations that justify moral classifications, which limits transparency and interpretability; and a predominant focus on English, which constrains the assessment of moral reasoning across diverse cultural settings. In this paper, we introduce MFTCXplain, a multilingual benchmark dataset for evaluating the moral reasoning of LLMs via multi-hop hate speech explanation using the Moral Foundations Theory. MFTCXplain comprises 3,000 tweets across Portuguese, Italian, Persian, and English, annotated with binary hate speech labels, moral categories, and text span-level rationales. Our results show a misalignment between LLM outputs and human annotations in moral reasoning tasks. While LLMs perform well in hate speech detection (F1 up to 0.836), their ability to predict moral sentiments is notably weak (F1 < 0.35). Furthermore, rationale alignment remains limited mainly in underrepresented languages. Our findings show the limited capacity of current LLMs to internalize and reflect human moral reasoning