# LLM-C3MOD: A Human-LLM Collaborative System for Cross-Cultural Hate Speech Moderation

arXiv: https://arxiv.org/abs/2503.07237 · HTML: https://mlsanity.com/paper/2503.07237/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 42 references that are in the map.
- First posted on arXiv 10 Mar 2025.

## Builds on (most cited first, 23 of 23)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection](https://mlsanity.com/paper/2012.10289/) (2020)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection](https://mlsanity.com/paper/2012.15761/) (2020)
- [Latent Hatred: A Benchmark for Understanding Implicit Hate Speech](https://mlsanity.com/paper/2109.05322/) (2021)
- [CONAN -- COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech](https://mlsanity.com/paper/1910.03270/) (2019)
- [Survey of Cultural Awareness in Language Models: Text and Beyond](https://mlsanity.com/paper/2411.00860/) (2024)
- [COLD: A Benchmark for Chinese Offensive Language Detection](https://mlsanity.com/paper/2201.06025/) (2022)
- [KoBBQ: Korean Bias Benchmark for Question Answering](https://mlsanity.com/paper/2307.16778/) (2023)
- [Probing LLMs for hate speech detection: strengths and vulnerabilities](https://mlsanity.com/paper/2310.12860/) (2023)
- [One-step and Two-step Classification for Abusive Language Detection on Twitter](https://mlsanity.com/paper/1706.01206/) (2017)
- [Evaluating GPT-3 Generated Explanations for Hateful Content Moderation](https://mlsanity.com/paper/2305.17680/) (2023)
- [Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysis](https://mlsanity.com/paper/2308.16705/) (2023)
- [HARE: Explainable Hate Speech Detection with Step-by-Step Reasoning](https://mlsanity.com/paper/2311.00321/) (2023)
- [Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Models](https://mlsanity.com/paper/2401.16727/) (2024)
- [Hate Personified: Investigating the role of LLMs in content moderation](https://mlsanity.com/paper/2410.02657/) (2024)
- [Listening to Affected Communities to Define Extreme Speech: Dataset and Experiments](https://mlsanity.com/paper/2203.11764/) (2022)
- [MemeGuard: An LLM and VLM-based Framework for Advancing Content Moderation via Meme Intervention](https://mlsanity.com/paper/2406.05344/) (2024)
- [Moderating New Waves of Online Hate with Chain-of-Thought Reasoning in Large Language Models](https://mlsanity.com/paper/2312.15099/) (2023)
- [Proactively Reducing the Hate Intensity of Online Posts via Hate Speech Normalization](https://mlsanity.com/paper/2206.04007/) (2022)
- ["I Searched for a Religious Song in Amharic and Got Sexual Content Instead": Investigating Online Harm in Low-Resourced Languages on YouTube](https://mlsanity.com/paper/2405.16656/) (2024)
- [LLM generated responses to mitigate the impact of hate speech](https://mlsanity.com/paper/2311.16905/) (2023)
- [A Federated Approach to Few-Shot Hate Speech Detection for Marginalized Communities](https://mlsanity.com/paper/2412.04942/) (2024)

## Built on by (most cited first, 5 of 5)

- [MoMoE: Mixture of Moderation Experts Framework for AI-Assisted Online Governance](https://mlsanity.com/paper/2505.14483/) (2025)
- [Label Over Logic? How Source Cues Bias Human Fallacy Judgments More Than LLMs](https://mlsanity.com/paper/2605.29928/) (2026)
- [Whitewashing Hate, Smearing Harmless Content: Annotator-Style Rebuttal Attacks on LLM-Based Moderation](https://mlsanity.com/paper/2608.22230/) (2026)
- [CC-Mediation: Evaluating Large Language Models for Cross-Cultural Conflict Mediation](https://mlsanity.com/paper/2609.04855/) (2026)
- [CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systems](https://mlsanity.com/paper/2609.22494/) (2026)

## Abstract (arXiv)

Content moderation is a global challenge, yet major tech platforms prioritize high-resource languages, leaving low-resource languages with scarce native moderators. Since effective moderation depends on understanding contextual cues, this imbalance increases the risk of improper moderation due to non-native moderators' limited cultural understanding. Through a user study, we identify that non-native moderators struggle with interpreting culturally-specific knowledge, sentiment, and internet culture in the hate speech moderation. To assist them, we present LLM-C3MOD, a human-LLM collaborative pipeline with three steps: (1) RAG-enhanced cultural context annotations; (2) initial LLM-based moderation; and (3) targeted human moderation for cases lacking LLM consensus. Evaluated on a Korean hate speech dataset with Indonesian and German participants, our system achieves 78% accuracy (surpassing GPT-4o's 71% baseline), while reducing human workload by 83.6%. Notably, human moderators excel at nuanced contents where LLMs struggle. Our findings suggest that non-native moderators, when properly supported by LLMs, can effectively contribute to cross-cultural hate speech moderation.