mlsanityData to 14 Oct 2026

KZ-SafetyPrompts: A Kazakh Safety Evaluation Prompt Dataset for Large Language Models

arXiv 2605.26947 ↗First posted 26 May 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Kazakh is underrepresented in resources for evaluating the safety behavior of large language models. We present KZ-SafetyPrompts, a Kazakh prompt dataset for safety evaluation across eleven categories covering common risk areas such as self-harm, violence, child exploitation, sexual content, racist content, radicalization, and regulated goods or illegal activities. The dataset contains 5,717 prompts written natively in Kazakh (Cyrillic), organized by category, with English translations for cross-lingual analysis. Prompts resemble realistic user queries, often in a teen or child style, and are phrased as intent prompts without procedural instructions. We document the writing protocol, labeling procedures (including borderline-case decision rules), and quality-control steps (schema standardization, completeness checks, and deduplication). We also align the categories with widely used safety taxonomies to support integration with existing evaluation pipelines. Baseline results with GPT-4o show an overall refusal rate of 28.2%, varying from 5.5% to 53.8% across categories, indicating that Kazakh prompts expose category-specific safety gaps not captured by English-only evaluation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 27 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

11 of 27 references.

References · 11

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  2. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  3. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  4. 2022Red Teaming Language Models with Language Modelscited by 1,026
  5. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  6. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 185
  7. 2023DICES Dataset: Diversity in Conversational AI Evaluation for Safetycited by 86
  8. 2025TUMLU: A Unified and Native Language Understanding Benchmark for Turkic Languagescited by 9
  9. 2024Recent Advancements and Challenges of Turkic Central Asian Language Processingcited by 6
  10. 2024Chinese Offensive Language Detection:Current Status and Future Directionscited by 5
  11. 2025LLM Safety for Childrencited by 5

Built on by

Not cited yet.