# ROK-FORTRESS: Measuring the Effect of Geopolitical Transcreation for National Security and Public Safety

arXiv: https://arxiv.org/abs/2605.14152 · HTML: https://mlsanity.com/paper/2605.14152/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 13 May 2026.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 12 of 12)

- [Multilingual Jailbreak Challenges in Large Language Models](https://mlsanity.com/paper/2310.06474/) (2023)
- [The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts](https://mlsanity.com/paper/2401.13136/) (2024)
- [AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policies](https://mlsanity.com/paper/2407.17436/) (2024)
- [The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm](https://mlsanity.com/paper/2406.18682/) (2024)
- [The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It](https://mlsanity.com/paper/2505.24119/) (2025)
- [CultureGuard: Towards Culturally-Aware Dataset and Guard Model for Multilingual Safety Applications](https://mlsanity.com/paper/2508.01710/) (2025)
- [FORTRESS: Frontier Risk Evaluation for National Security and Public Safety](https://mlsanity.com/paper/2506.14922/) (2025)
- [LinguaSafe: A Comprehensive Multilingual Safety Benchmark for Large Language Models](https://mlsanity.com/paper/2508.12733/) (2025)
- [SafeWorld: Geo-Diverse Safety Alignment](https://mlsanity.com/paper/2412.06483/) (2024)
- [Measuring What Matters: A Framework for Evaluating Safety Risks in Real-World LLM Applications](https://mlsanity.com/paper/2507.09820/) (2025)
- [The Tower of Babel Revisited: Multilingual Jailbreak Prompts on Closed-Source Large Language Models](https://mlsanity.com/paper/2505.12287/) (2025)
- [CAGE: A Framework for Culturally Adaptive Red-Teaming Benchmark Generation](https://mlsanity.com/paper/2602.20170/) (2026)

## Abstract (arXiv)

Safety evaluations for large language models (LLMs) increasingly target high-stakes National Security and Public Safety (NSPS) risks, yet multilingual safety is mostly assessed through translation-only benchmarks that preserve the underlying scenario, leaving how language and geopolitical context interact largely unexamined beyond a few language pairs. We introduce ROK-FORTRESS, a bilingual, culturally adversarial NSPS benchmark that uses the English-Korean language pair and U.S.-ROK geopolitical axis as a case study, separating the effects of language and geopolitical grounding via a transcreation matrix: adversarial intents are evaluated under controlled combinations of (i) English versus Korean language and (ii) U.S. versus Korean entities, institutions, and operational details. Each adversarial prompt is paired with a dual-use benign counterpart to quantify over-refusal, and responses are scored by calibrated LLM-as-a-judge panels using expert-crafted, prompt-specific binary rubrics. Across a dual-track set of frontier and Korean-optimized models, we find a consistent suppression effect in Korean variants and substantial model-to-model variation in how geopolitical grounding interacts with language; in a subset of models, Korean grounding further mitigates the language-driven suppression. This indicates that, at least in the English-Korean case, safety behavior is shaped by language-as-risk signals and context interactions that translation-only evaluations miss. A direct-request ablation that strips jailbreak wrappers separates a small but persistent reduction for closed-source models from a larger, wrapper-dependent effect that reverses for open-source models, suggesting part of the Korean suppression reflects prompt specialization rather than intrinsic language-based safety alignment. The transcreation matrix methodology is designed to generalize to other language-culture pairs.