# LionGuard: Building a Contextualized Moderation Classifier to Tackle Localized Unsafe Content

arXiv: https://arxiv.org/abs/2407.10995 · HTML: https://mlsanity.com/paper/2407.10995/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 49 references that are in the map.
- First posted on arXiv 24 Jun 2024.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [A Holistic Approach to Undesired Content Detection in the Real World](https://mlsanity.com/paper/2208.03274/) (2022)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://mlsanity.com/paper/1903.08983/) (2019)
- [HateBERT: Retraining BERT for Abusive Language Detection in English](https://mlsanity.com/paper/2010.12472/) (2020)
- [Hate Speech Dataset from a White Supremacy Forum](https://mlsanity.com/paper/1809.04444/) (2018)
- [A New Generation of Perspective API: Efficient Multilingual Character-level Transformers](https://mlsanity.com/paper/2202.11176/) (2022)
- [A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media](https://mlsanity.com/paper/1910.12574/) (2019)
- [Detecting Hate Speech with GPT-3](https://mlsanity.com/paper/2103.12407/) (2021)
- [Disentangling Hate in Online Memes](https://mlsanity.com/paper/2108.06207/) (2021)
- [Detecting weak and strong Islamophobic hate speech on social media](https://mlsanity.com/paper/1812.10400/) (2018)
- [TotalDefMeme: A Multi-Attribute Meme dataset on Total Defence in Singapore](https://mlsanity.com/paper/2305.17911/) (2023)
- [Developing a concept-level knowledge base for sentiment analysis in Singlish](https://mlsanity.com/paper/1707.04408/) (2017)

## Built on by (most cited first, 5 of 5)

- [WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Models](https://mlsanity.com/paper/2408.03837/) (2024)
- [Safe at the Margins: A General Approach to Safety Alignment in Low-Resource English Languages -- A Singlish Case Study](https://mlsanity.com/paper/2502.12485/) (2025)
- [A Flexible Large Language Models Guardrail Development Methodology Applied to Off-Topic Prompt Detection](https://mlsanity.com/paper/2411.12946/) (2024)
- [Toxicity-Aware Few-Shot Prompting for Low-Resource Singlish Translation](https://mlsanity.com/paper/2507.11966/) (2025)
- [Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models](https://mlsanity.com/paper/2502.18101/) (2025)

## Abstract (arXiv)

As large language models (LLMs) become increasingly prevalent in a wide variety of applications, concerns about the safety of their outputs have become more significant. Most efforts at safety-tuning or moderation today take on a predominantly Western-centric view of safety, especially for toxic, hateful, or violent speech. In this paper, we describe LionGuard, a Singapore-contextualized moderation classifier that can serve as guardrails against unsafe LLM outputs. When assessed on Singlish data, LionGuard outperforms existing widely-used moderation APIs, which are not finetuned for the Singapore context, by 14% (binary) and up to 51% (multi-label). Our work highlights the benefits of localization for moderation classifiers and presents a practical and scalable approach for low-resource languages.