mlsanityData to 7 Oct 2026

Multilingual Jailbreak Challenges in Large Language Models

arXiv 2310.06474 ↗First posted 10 Oct 2023cs.CLcited by 222

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

While large language models (LLMs) exhibit remarkable capabilities across a wide range of tasks, they pose potential safety concerns, such as the ``jailbreak'' problem, wherein malicious instructions can manipulate LLMs to exhibit undesirable behavior. Although several preventive measures have been developed to mitigate the potential risks associated with LLMs, they have primarily focused on English. In this study, we reveal the presence of multilingual jailbreak challenges within LLMs and consider two potential risky scenarios: unintentional and intentional. The unintentional scenario involves users querying LLMs using non-English prompts and inadvertently bypassing the safety mechanisms, while the intentional scenario concerns malicious users combining malicious instructions with multilingual prompts to deliberately attack LLMs. The experimental results reveal that in the unintentional scenario, the rate of unsafe content increases as the availability of languages decreases. Specifically, low-resource languages exhibit about three times the likelihood of encountering harmful content compared to high-resource languages, with both ChatGPT and GPT-4. In the intentional scenario, multilingual prompts can exacerbate the negative impact of malicious instructions, with astonishingly high rates of unsafe output: 80.92% for ChatGPT and 40.71% for GPT-4. To handle such a challenge in the multilingual context, we propose a novel textscSelf-Defense framework that automatically generates multilingual training data for safety fine-tuning. Experimental results show that ChatGPT fine-tuned with such data can achieve a substantial reduction in unsafe content generation. Data is available at urlhttps://github.com/DAMO-NLP-SG/multilingual-safety-for-LLMs.

Key facts. Cited by 222 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 30 of its 39 references that are in the map. First posted on arXiv 10 Oct 2023.

Builds on

30 of 39 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  2. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  4. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
  5. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 25

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  8. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  9. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  10. 2022Red Teaming Language Models with Language Modelscited by 983
  11. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  12. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
13 more
  1. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  2. 2023Is ChatGPT a General-Purpose Natural Language Processing Task Solver?cited by 486
  3. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  4. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  5. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  6. 2023How is ChatGPT's behavior changing over time?cited by 265
  7. 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 262
  8. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  9. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 203
  10. 2021Challenges in Detoxifying Language Modelscited by 169
  11. 2023M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Modelscited by 109
  12. 2021Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoningcited by 64
  13. 2023Exploring New Frontiers in Agricultural NLP: Investigating the Potential of Large Language Models for Food Applicationscited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 82; 2025: 73; 2026: 63.

222 papers.

Sort:

By area:

  1. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  2. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  3. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  4. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  5. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  6. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  7. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 160
  8. 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 158
  9. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 149
  10. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 135
  11. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 121
  12. 2024Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontiercited by 121
38 more
  1. 2023SeaLLMs -- Large Language Models for Southeast Asiacited by 104
  2. 2023Evil Geniuses: Delving into the Safety of LLM-based Agentscited by 103
  3. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 100
  4. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstructioncited by 95
  5. 2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templatescited by 94
  6. 2024JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMscited by 87
  7. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  8. 2025Do Multilingual LLMs Think In English?cited by 75
  9. 2024Safeguarding Large Language Models: A Surveycited by 73
  10. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  11. 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 69
  12. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 68
  13. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 68
  14. 2024The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harmcited by 67
  15. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  16. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  17. 2024Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligencecited by 65
  18. 2024RigorLLM: Resilient Guardrails for Large Language Models against Undesired Contentcited by 65
  19. 2024Fight Back Against Jailbreaking via Prompt Adversarial Tuningcited by 63
  20. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  21. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
  22. 2025PolyGuard: A Multilingual Safety Moderation Tool for 17 Languagescited by 57
  23. 2025Thought Crime: Backdoors and Emergent Misalignment in Reasoning Modelscited by 57
  24. 2024Risks of AI Scientists: Prioritizing Safeguarding Over Autonomycited by 49
  25. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 47
  26. 2024Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Modelscited by 44
  27. 2024JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Frameworkcited by 44
  28. 2024A Theoretical Understanding of Self-Correction through In-context Alignmentcited by 43
  29. 2024RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMscited by 42
  30. 2024Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Ratescited by 42
  31. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistencycited by 42
  32. 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating Itcited by 42
  33. 2024Voice Jailbreak Attacks Against GPT-4ocited by 39
  34. 2024RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agentcited by 39
  35. 2025Refusal Direction is Universal Across Safety-Aligned Languagescited by 37
  36. 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Modelscited by 34
  37. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  38. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 33

50 most cited of 222 All 222 (JSON)