mlsanityData to 13 Oct 2026

Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions

arXiv 2502.04322 ↗First posted 6 Feb 2025cs.LG cs.AI cs.CLcited by 15

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Despite extensive safety alignment efforts, large language models (LLMs) remain vulnerable to jailbreak attacks that elicit harmful behavior. While existing studies predominantly focus on attack methods that require technical expertise, two critical questions remain underexplored: (1) Are jailbroken responses truly useful in enabling average users to carry out harmful actions? (2) Do safety vulnerabilities exist in more common, simple human-LLM interactions? In this paper, we demonstrate that LLM responses most effectively facilitate harmful actions when they are both actionable and informative--two attributes easily elicited in multi-step, multilingual interactions. Using this insight, we propose HarmScore, a jailbreak metric that measures how effectively an LLM response enables harmful actions, and Speak Easy, a simple multi-step, multilingual attack framework. Notably, by incorporating Speak Easy into direct request and jailbreak baselines, we see an average absolute increase of 0.319 in Attack Success Rate and 0.426 in HarmScore in both open-source and proprietary LLMs across four safety benchmarks. Our work reveals a critical yet often overlooked vulnerability: Malicious users can easily exploit common interaction patterns for harmful intentions.

Key facts. Cited by 15 papers in the map (data through 13 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 51 of its 71 references that are in the map. First posted on arXiv 6 Feb 2025.

Builds on

51 of 71 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deep10 Jun
  2. 2024The Language Barrier: Dissecting Safety Challenges of LLMs in Multilingual Contexts23 Jan
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  4. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned23 Aug

Other references · 47

  1. 2023GPT-4 Technical Reportcited by 18,278
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,072
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,376
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,986
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,348
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 3,999
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,691
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,011
  9. 2024Qwen2 Technical Reportcited by 2,102
  10. 2024Mixtral of Expertscited by 1,811
  11. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,637
  12. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,405
35 more
  1. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,404
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,404
  3. 2022Red Teaming Language Models with Language Modelscited by 1,014
  4. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 878
  5. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 792
  6. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 680
  7. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 664
  8. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 616
  9. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 556
  10. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 531
  11. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 488
  12. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 458
  13. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 448
  14. 2024A StrongREJECT for Empty Jailbreakscited by 379
  15. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 354
  16. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  17. 2023Low-Resource Languages Jailbreak GPT-4cited by 306
  18. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 294
  19. 2023MEGA: Multilingual Evaluation of Generative AIcited by 293
  20. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 235
  21. 2023An Overview of Catastrophic AI Riskscited by 219
  22. 2023ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learningcited by 205
  23. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 193
  24. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 171
  25. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 151
  26. 2024Arithmetic Control of LLMs for Diverse User Preferences: Directional Preference Alignment with Multi-Objective Rewardscited by 148
  27. 2020Unsupervised Question Decomposition for Question Answeringcited by 118
  28. 2022Successive Prompting for Decomposing Complex Questionscited by 94
  29. 2024Does Refusal Training in LLMs Generalize to the Past Tense?cited by 79
  30. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 70
  31. 2024The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Modelscited by 62
  32. 2021Coherence boosting: When your pretrained language model is not paying enough attentioncited by 38
  33. 2024WildVis: Open Source Visualizer for Million-Scale Chat Logs in the Wildcited by 20
  34. 2024Endless Jailbreaks with Bijection Learningcited by 18
  35. 2022Polyglot Prompt: Multilingual Multitask PrompTrainingcited by 6

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 10; 2026: 5.

15 papers.

Sort:

By area:

  1. 2025The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating Itcited by 43
  2. 2025Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Modelscited by 16
  3. 2025Where to Start Alignment? Diffusion Large Language Model May Demand a Distinct Positioncited by 11
  4. 2025When Style Breaks Safety: Defending LLMs Against Superficial Style Alignmentcited by 6
  5. 2025Jailbreak Distillation: Renewable Safety Benchmarkingcited by 5
  6. 2025CoBia: Constructed Conversations Can Trigger Otherwise Concealed Societal Biases in LLMscited by 5
  7. 2025Chain-of-Code Collapse: Reasoning Failures in LLMs via Adversarial Prompting in Code Generationcited by 4
  8. 2025MetaCipher: A Time-Persistent and Universal Multi-Agent Framework for Cipher-Based Jailbreak Attacks for LLMscited by 3
  9. 2025Jailbreaking Large Language Models Through Content Concretizationcited by 2
  10. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  11. 2026Jailbroken Frontier Models Retain Their Capabilitiescited by 1
  12. 2026Jailbreaking LLMs via Calibrationcited by 0
3 more
  1. 2026Jailbreaking for the Average Jane: Choosing Optimal Jailbreaks via Bandit Algorithms for Automatically Enhanced Queriescited by 0
  2. 2026MJ: Multi-turn LLM Jailbreaking via Decomposed Credit Assignmentcited by 0
  3. 2026IndicSafeEval: Safety Robustness of Large Language Models under Multilingual Persuasive Jailbreak Attackscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysis
  2. 2025Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs
  3. 2024Characterizing and Evaluating the Reliability of LLMs against Jailbreak Attacks
  4. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models
  5. 2025You Can't Eat Your Cake and Have It Too: The Performance Degradation of LLMs with Jailbreak Defense
  6. 2025Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures
  7. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding
  8. 2023Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks