# Guardians of Discourse: Evaluating LLMs on Multilingual Offensive Language Detection

arXiv: https://arxiv.org/abs/2410.15623 · HTML: https://mlsanity.com/paper/2410.15623/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 21 Oct 2024.

## Builds on (most cited first, 16 of 16)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech](https://mlsanity.com/paper/2302.07736/) (2023)
- [Extrapolating Large Language Models to Non-English by Aligning Languages](https://mlsanity.com/paper/2308.04948/) (2023)
- [SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification](https://mlsanity.com/paper/2004.14454/) (2020)
- [Deep Learning Models for Multilingual Hate Speech Detection](https://mlsanity.com/paper/2004.06465/) (2020)
- [When does In-context Learning Fall Short and Why? A Study on Specification-Heavy Tasks](https://mlsanity.com/paper/2311.08993/) (2023)
- [Large Language Models in Mental Health Care: a Scoping Review](https://mlsanity.com/paper/2401.02984/) (2024)
- [HARE: Explainable Hate Speech Detection with Step-by-Step Reasoning](https://mlsanity.com/paper/2311.00321/) (2023)
- [Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation](https://mlsanity.com/paper/1909.00437/) (2019)
- [Generating Valid and Natural Adversarial Examples with Large Language Models](https://mlsanity.com/paper/2311.11861/) (2023)
- [Model-Agnostic Meta-Learning for Multilingual Hate Speech Detection](https://mlsanity.com/paper/2303.02513/) (2023)
- [Cross-lingual Capsule Network for Hate Speech Detection in Social Media](https://mlsanity.com/paper/2108.03089/) (2021)
- [Thinker-DDM: Modeling Deliberation for Machine Translation with a Drift-Diffusion Process](https://mlsanity.com/paper/2402.10699/) (2024)

## Built on by (most cited first, 5 of 5)

- [Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement](https://mlsanity.com/paper/2507.07640/) (2025)
- [Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Study](https://mlsanity.com/paper/2505.06149/) (2025)
- [MedFact: A Large-scale Chinese Dataset for Evidence-based Medical Fact-checking of LLM Responses](https://mlsanity.com/paper/2509.17436/) (2025)
- [Artificial Conversations, Real Results: Fostering Language Detection with Synthetic Data](https://mlsanity.com/paper/2503.24062/) (2025)
- [Subasa - Adapting Language Models for Low-resourced Offensive Language Detection in Sinhala](https://mlsanity.com/paper/2504.02178/) (2025)

## Abstract (arXiv)

Identifying offensive language is essential for maintaining safety and sustainability in the social media era. Though large language models (LLMs) have demonstrated encouraging potential in social media analytics, they lack thorough evaluation when in offensive language detection, particularly in multilingual environments. We for the first time evaluate multilingual offensive language detection of LLMs in three languages: English, Spanish, and German with three LLMs, GPT-3.5, Flan-T5, and Mistral, in both monolingual and multilingual settings. We further examine the impact of different prompt languages and augmented translation data for the task in non-English contexts. Furthermore, we discuss the impact of the inherent bias in LLMs and the datasets in the mispredictions related to sensitive topics.