mlsanityData to 7 Oct 2026

Low-Resource Languages Jailbreak GPT-4

arXiv 2310.02446 ↗First posted 3 Oct 2023cs.CL cs.AI cs.CRcited by 290

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

AI safety training and red-teaming of large language models (LLMs) are measures to mitigate the generation of unsafe content. Our work exposes the inherent cross-lingual vulnerability of these safety mechanisms, resulting from the linguistic inequality of safety training data, by successfully circumventing GPT-4's safeguard through translating unsafe English inputs into low-resource languages. On the AdvBenchmark, GPT-4 engages with the unsafe translated inputs and provides actionable items that can get the users towards their harmful goals 79% of the time, which is on par with or even surpassing state-of-the-art jailbreaking attacks. Other high-/mid-resource languages have significantly lower attack success rate, which suggests that the cross-lingual vulnerability mainly applies to low-resource languages. Previously, limited training on low-resource languages primarily affects speakers of those languages, causing technological disparities. However, our work highlights a crucial shift: this deficiency now poses a risk to all LLMs users. Publicly available translation APIs enable anyone to exploit LLMs' safety vulnerabilities. Therefore, our work calls for a more holistic red-teaming efforts to develop robust multilingual safeguards with wide language coverage.

Key facts. Cited by 290 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 44 of its 53 references that are in the map. First posted on arXiv 3 Oct 2023.

Builds on

44 of 53 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Models4 Sep
  2. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher12 Aug
  3. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  5. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
  6. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 38

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  7. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  8. 2022Red Teaming Language Models with Language Modelscited by 983
  9. 2021Ethical and social risks of harm from Language Modelscited by 957
  10. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  11. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  12. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 789
26 more
  1. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  2. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  3. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  4. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  5. 2023Are aligned neural networks adversarially aligned?cited by 305
  6. 2023How is ChatGPT's behavior changing over time?cited by 265
  7. 2019Evaluating Gender Bias in Machine Translationcited by 247
  8. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 203
  9. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  10. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 176
  11. 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 171
  12. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  13. 2023SeamlessM4T: Massively Multilingual & Multimodal Machine Translationcited by 124
  14. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
  15. 2019CONAN -- COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speechcited by 109
  16. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 86
  17. 2022Building Machine Translation Systems for the Next Thousand Languagescited by 69
  18. 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Modelscited by 68
  19. 2023Black Box Adversarial Prompting for Foundation Modelscited by 62
  20. 2023All Languages Matter: On the Multilingual Safety of Large Language Modelscited by 51
  21. 2020Universal Adversarial Attacks with Natural Triggers for Text Classificationcited by 50
  22. 2021Mitigating harm in language models with conditional-likelihood filtrationcited by 38
  23. 2023ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languagescited by 33
  24. 2022Why So Toxic? Measuring and Triggering Toxic Behavior in Open-Domain Chatbotscited by 31
  25. 2022Square One Bias in NLP: Towards a Multi-Dimensional Exploration of the Research Manifoldcited by 29
  26. 2023CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Modelscited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 121; 2025: 81; 2026: 80.

290 papers.

Sort:

By area:

  1. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
  2. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  3. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  4. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  5. 2024A StrongREJECT for Empty Jailbreakscited by 352
  6. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  7. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  8. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  9. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  10. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 189
  11. 2024Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuningcited by 174
  12. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 160
38 more
  1. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  2. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 153
  3. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  4. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  5. 2024Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Modelscited by 122
  6. 2024Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontiercited by 121
  7. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  8. 2024Automatic and Universal Prompt Injection Attacks against Large Language Modelscited by 112
  9. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  10. 2023Evil Geniuses: Delving into the Safety of LLM-based Agentscited by 103
  11. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 98
  12. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 95
  13. 2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templatescited by 94
  14. 2024Coercing LLMs to do and reveal (almost) anythingcited by 88
  15. 2024JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMscited by 87
  16. 2024Defending Against Unforeseen Failure Modes with Latent Adversarial Trainingcited by 82
  17. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  18. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinkingcited by 78
  19. 2024Safeguarding Large Language Models: A Surveycited by 73
  20. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  21. 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothingcited by 68
  22. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 68
  23. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  24. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  25. 2024RigorLLM: Resilient Guardrails for Large Language Models against Undesired Contentcited by 65
  26. 2024Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapescited by 63
  27. 2024Building Guardrails for Large Language Modelscited by 61
  28. 2024Get my drift? Catching LLM Task Drift with Activation Deltascited by 61
  29. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
  30. 2024Query-Based Adversarial Prompt Generationcited by 55
  31. 2023Privacy in Large Language Models: Attacks, Defenses and Future Directionscited by 48
  32. 2024PAL: Proxy-Guided Black-Box Attack on Large Language Modelscited by 46
  33. 2024A Safe Harbor for AI Evaluation and Red Teamingcited by 46
  34. 2024Phantom: General Backdoor Attacks on Retrieval Augmented Language Generationcited by 46
  35. 2024A Theoretical Understanding of Self-Correction through In-context Alignmentcited by 43
  36. 2024Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Modelscited by 42
  37. 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating Itcited by 42
  38. 2024Distract Large Language Models for Automatic Jailbreak Attackcited by 40

50 most cited of 290 All 290 (JSON)