TukaBench: A Culturally Grounded Jailbreak Benchmark for African Languages
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
Safety evaluation of Large Language Models (LLMs) remains heavily English-centric, leaving Low-Resource Languages (LRLs), particularly African ones, critically underexplored. We introduce TUKABENCH, a jailbreak benchmark for seven African languages that extends JailbreakBench (JBB) beyond direct translation through four settings: human translation of JBB prompts, English adaptation to African contexts followed by human translation, human-curated prompts validated through interactions with GPT-5.2, and code-switched prompts combining English and African languages, isolating the effect of language, cultural grounding, and prompt evasiveness on model safety. Across closed and open models, prompting in African languages reduces refusal relative to English, with culturally adapted prompts leading to least refusal. The evaluation also surfaces two structural limitations: model comprehension failures and reduced LLM-as-a-judge reliability in LRLs. To capture the first, we introduce Deflection alongside Refused and Jailbroken; to assess the second, we validate outputs with human annotations, showing that judge-human agreement drops in lower-resource languages and less commonly supported scripts.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 20 of its 32 references that are in the map. First posted on arXiv 31 May 2026.
Builds on
20 of 32 references.
Closest ancestors · 5
Same direction, newest first
- 2026One Word at a Time: Incremental Completion Decomposition Breaks LLM Safety1 Apr
- 2026Boundary Point Jailbreaking of Black-Box LLMs16 Feb
- 2026Break Me If You Can: Self-Jailbreaking of Aligned LLMs via Lexical Insertion Prompting6 Jan
- 2025Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?1 Nov
- 2023Low-Resource Languages Jailbreak GPT-43 Oct
Other references · 15
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
- 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 97
- 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 96
- 2023Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?cited by 87
- 2025PolyGuard: A Multilingual Safety Moderation Tool for 17 Languagescited by 57
- 2024Code-Switching Red-Teaming: LLM Evaluation for Safety and Multilingual Understandingcited by 34
- 2024Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixturecited by 22
- 2025The Token Tax: Systematic Bias in Multilingual Tokenizationcited by 13
- 2026AfriqueLLM: How Data Mixing and Model Architecture Impact Continued Pre-training for African Languagescited by 12
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Challenges and Recommendations for LLM-as-a-Judge in Multilingual Settings and for Low-Resource Languagescited by 1
- 2026SpeechJBB: Probing Safety Alignment and Comprehension in Large Audio Language Models under Code-Switched Speechcited by 0
- 2026Language-Specific Gaps in AI Safety Training Datasetscited by 0
- 2026Language-Specific Gaps in AI Safety Training Datasets13 Aug 2026
- 2026Challenges and Recommendations for LLM-as-a-Judge in Multilingual Settings and for Low-Resource Languages2 Jul 2026
- 2026SpeechJBB: Probing Safety Alignment and Comprehension in Large Audio Language Models under Code-Switched Speech4 Jun 2026