IndoSafety: Culturally Grounded Safety for LLMs in Indonesian Languages
Not on the map. Why?
Abstract (arXiv)
Although region-specific large language models (LLMs) are increasingly developed, their safety remains underexplored, particularly in culturally diverse settings like Indonesia, where sensitivity to local norms is essential and highly valued by the community. In this work, we present IndoSafety, the first high-quality, human-verified safety evaluation dataset tailored for the Indonesian context, covering five language varieties: formal and colloquial Indonesian, along with three major local languages: Javanese, Sundanese, and Minangkabau. IndoSafety is constructed by extending prior safety frameworks to develop a taxonomy that captures Indonesia's sociocultural context. We find that existing Indonesian-centric LLMs often generate unsafe outputs, particularly in colloquial and local language settings, while fine-tuning on IndoSafety significantly improves safety while preserving task performance. Our work highlights the critical need for culturally grounded safety evaluation and provides a concrete step toward responsible LLM deployment in multilingual settings. Warning: This paper contains example data that may be offensive, harmful, or biased.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 23 of its 44 references that are in the map. First posted on arXiv 3 Jun 2025.
Builds on
23 of 44 references.
References · 23
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
- 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
- 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
- 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
- 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
- 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 247
11 more
- 2022Are Large Pre-Trained Language Models Leaking Your Personal Information?cited by 200
- 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 192
- 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 170
- 2024Defending LLMs against Jailbreaking Attacks via Backtranslationcited by 70
- 2023Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLUcited by 53
- 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspectivecited by 48
- 2022SafeText: A Benchmark for Exploring Physical Safety in Language Modelscited by 41
- 2024IndoCulture: Exploring Geographically-Influenced Cultural Commonsense Reasoning Across Eleven Indonesian Provincescited by 39
- 2023COPAL-ID: Indonesian Language Reasoning with Local Culture and Nuancescited by 35
- 2024Arabic Dataset for LLM Safeguard Evaluationcited by 17
- 2024Cracking the Code: Multi-domain LLM Evaluation on Real-World Professional Exams in Indonesiacited by 4
Built on by
Citing papers per year: 2025: 1; 2026: 6.
7 papers.
- 2026IndicJR: A Judge-Free Benchmark of Jailbreak Robustness in South Asian Languagescited by 5
- 2026IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asiacited by 3
- 2025Role-Aware Language Models for Secure and Contextualized Access Control in Organizationscited by 2
- 2026Low-Resource Safety Failures Are Action Failures, Not Representation Failurescited by 2
- 2026AdaCultureSafe: Adaptive Cultural Safety Grounded by Cultural Knowledge in Large Language Modelscited by 0
- 2026LLM Safety Alignment in Low-Resource Languages: A Systematic Literature Reviewcited by 0
- 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0
- 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systems18 Sep 2026
- 2026LLM Safety Alignment in Low-Resource Languages: A Systematic Literature Review20 Jul 2026
- 2026Low-Resource Safety Failures Are Action Failures, Not Representation Failures31 May 2026
- 2026IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia18 Mar 2026
- 2026AdaCultureSafe: Adaptive Cultural Safety Grounded by Cultural Knowledge in Large Language Models9 Mar 2026
- 2026IndicJR: A Judge-Free Benchmark of Jailbreak Robustness in South Asian Languages18 Feb 2026
- 2025Role-Aware Language Models for Secure and Contextualized Access Control in Organizations31 Jul 2025