# The Tower of Babel Revisited: Multilingual Jailbreak Prompts on Closed-Source Large Language Models

arXiv: https://arxiv.org/abs/2505.12287 · HTML: https://mlsanity.com/paper/2505.12287/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 29 of its 55 references that are in the map.
- First posted on arXiv 18 May 2025.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 29 of 29)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://mlsanity.com/paper/2402.04249/) (2024)
- [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023)
- [Frontier Models are Capable of In-context Scheming](https://mlsanity.com/paper/2412.04984/) (2024)
- [Jailbreak Attacks and Defenses Against Large Language Models: A Survey](https://mlsanity.com/paper/2407.04295/) (2024)
- [The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1](https://mlsanity.com/paper/2502.12659/) (2025)
- [Red-Teaming for Generative AI: Silver Bullet or Security Theater?](https://mlsanity.com/paper/2401.15897/) (2024)
- [Safeguarding Large Language Models: A Survey](https://mlsanity.com/paper/2406.02622/) (2024)
- [A Complete Survey on LLM-based AI Chatbots](https://mlsanity.com/paper/2406.16937/) (2024)
- [Jailbreak Large Vision-Language Models Through Multi-Modal Linkage](https://mlsanity.com/paper/2412.00473/) (2024)
- [All Languages Matter: On the Multilingual Safety of Large Language Models](https://mlsanity.com/paper/2310.00905/) (2023)
- [COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning](https://mlsanity.com/paper/2403.18058/) (2024)
- [Jailbreak Attacks and Defenses against Multimodal Generative Models: A Survey](https://mlsanity.com/paper/2411.09259/) (2024)
- [UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generation](https://mlsanity.com/paper/2311.15296/) (2023)
- [Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture](https://mlsanity.com/paper/2407.07342/) (2024)
- [Understanding and Enhancing the Transferability of Jailbreaking Attacks](https://mlsanity.com/paper/2502.03052/) (2025)
- [Sandwich attack: Multi-language Mixture Adaptive Attack on LLMs](https://mlsanity.com/paper/2404.07242/) (2024)
- [SequentialBreak: Large Language Models Can be Fooled by Embedding Jailbreak Prompts into Sequential Prompt Chains](https://mlsanity.com/paper/2411.06426/) (2024)
- [The Open Source Advantage in Large Language Models (LLMs)](https://mlsanity.com/paper/2412.12004/) (2024)
- [Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patching](https://mlsanity.com/paper/2405.13820/) (2024)
- [Look Within, Why LLMs Hallucinate: A Causal Perspective](https://mlsanity.com/paper/2407.10153/) (2024)
- [Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferences](https://mlsanity.com/paper/2502.08142/) (2025)
- [Evolving Security in LLMs: A Study of Jailbreak Attacks and Defenses](https://mlsanity.com/paper/2504.02080/) (2025)
- [`From Prompt to Perturbation': An Adaptive Framework for Voice-Based Jailbreaks on Audio LLMs](https://mlsanity.com/paper/2502.00735/) (2025)
- [Bridging the LLM Accessibility Divide? Performance, Fairness, and Cost of Closed versus Open LLMs for Automated Essay Scoring](https://mlsanity.com/paper/2503.11827/) (2025)
- [Safety Evaluation and Enhancement of DeepSeek Models in Chinese Contexts](https://mlsanity.com/paper/2503.16529/) (2025)

## Built on by (most cited first, 3 of 3)

- [Evaluating the Robustness of Large Language Model Safety Guardrails Against Adversarial Attacks](https://mlsanity.com/paper/2511.22047/) (2025)
- [ROK-FORTRESS: Measuring the Effect of Geopolitical Transcreation for National Security and Public Safety](https://mlsanity.com/paper/2605.14152/) (2026)
- [Safety Targeted Embedding Exploit via Refinement](https://mlsanity.com/paper/2607.01859/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have seen widespread applications across various domains, yet remain vulnerable to adversarial prompt injections. While most existing research on jailbreak attacks and hallucination phenomena has focused primarily on open-source models, we investigate the frontier of closed-source LLMs under multilingual attack scenarios. We present a first-of-its-kind integrated adversarial framework that leverages diverse attack techniques to systematically evaluate frontier proprietary solutions, including GPT-4o, DeepSeek-R1, Gemini-1.5-Pro, and Qwen-Max. Our evaluation spans six categories of security contents in both English and Chinese, generating 38,400 responses across 32 types of jailbreak attacks. Attack success rate (ASR) is utilized as the quantitative metric to assess performance from three dimensions: prompt design, model architecture, and language environment. Our findings suggest that Qwen-Max is the most vulnerable, while GPT-4o shows the strongest defense. Notably, prompts in Chinese consistently yield higher ASRs than their English counterparts, and our novel Two-Sides attack technique proves to be the most effective across all models. This work highlights a dire need for language-aware alignment and robust cross-lingual defenses in LLMs, and we hope it will inspire researchers, developers, and policymakers toward more robust and inclusive AI systems.