mlsanityData to 7 Oct 2026

Does Refusal Training in LLMs Generalize to the Past Tense?

arXiv 2407.11969 ↗First posted 16 Jul 2024cs.CL cs.AI cs.LGcited by 74

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Refusal training is widely used to prevent LLMs from generating harmful, undesirable, or illegal outputs. We reveal a curious generalization gap in the current refusal training approaches: simply reformulating a harmful request in the past tense (e.g., "How to make a Molotov cocktail?" to "How did people make a Molotov cocktail?") is often sufficient to jailbreak many state-of-the-art LLMs. We systematically evaluate this method on Llama-3 8B, Claude-3.5 Sonnet, GPT-3.5 Turbo, Gemma-2 9B, Phi-3-Mini, GPT-4o mini, GPT-4o, o1-mini, o1-preview, and R2D2 models using GPT-3.5 Turbo as a reformulation model. For example, the success rate of this simple attack on GPT-4o increases from 1% using direct requests to 88% using 20 past tense reformulation attempts on harmful requests from JailbreakBench with GPT-4 as a jailbreak judge. Interestingly, we also find that reformulations in the future tense are less effective, suggesting that refusal guardrails tend to consider past historical questions more benign than hypothetical future questions. Moreover, our experiments on fine-tuning GPT-3.5 Turbo show that defending against past reformulations is feasible when past tense examples are explicitly included in the fine-tuning data. Overall, our findings highlight that the widely used alignment techniques -- such as SFT, RLHF, and adversarial training -- employed to align the studied models can be brittle and do not always generalize as intended. We provide code and jailbreak artifacts at https://github.com/tml-epfl/llm-past-tense.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 31 of its 44 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

31 of 44 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Improving Alignment and Robustness with Circuit Breakers6 Jun

Other references · 30

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  9. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  10. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  11. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  12. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
18 more
  1. 2022Red Teaming Language Models with Language Modelscited by 983
  2. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  3. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  4. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  5. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  6. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  7. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
  8. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  9. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  10. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  11. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
  12. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 128
  13. 2024Query-Based Adversarial Prompt Generationcited by 55
  14. 2023Managing extreme AI risks amid rapid progresscited by 48
  15. 2024PAL: Proxy-Guided Black-Box Attack on Large Language Modelscited by 46
  16. 2024RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMscited by 42
  17. 2024Preference Tuning For Toxicity Mitigation Generalizes Across Languagescited by 31
  18. 2024ChatGPT Doesn't Trust Chargers Fans: Guardrail Sensitivity in Contextcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 41; 2026: 22.

74 papers.

Sort:

By area:

  1. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  2. 2025Trading Inference-Time Compute for Adversarial Robustnesscited by 60
  3. 2025Safety in Large Reasoning Models: A Surveycited by 58
  4. 2025AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integrationcited by 38
  5. 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreakingcited by 34
  6. 2025Representation Bending for Large Language Model Safetycited by 28
  7. 2025Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agentscited by 23
  8. 2024Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language Modelscited by 21
  9. 2024Endless Jailbreaks with Bijection Learningcited by 18
  10. 2024WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Modelscited by 17
  11. 2025LLM-Safety Evaluations Lack Robustnesscited by 16
  12. 2025Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expressioncited by 16
38 more
  1. 2025Safety Reasoning with Guidelinescited by 14
  2. 2025Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactionscited by 14
  3. 2025AlphaAlign: Incentivizing Safety Alignment with Extremely Simplified Reinforcement Learningcited by 14
  4. 2024You Know What I'm Saying: Jailbreak Attack via Implicit Referencecited by 13
  5. 2025Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulationcited by 13
  6. 2025Lifelong Safety Alignment for Language Modelscited by 13
  7. 2026Prompt Injection as Role Confusioncited by 11
  8. 2024AmpleGCG-Plus: A Strong Generative Model of Adversarial Suffixes to Jailbreak LLMs with Higher Success Rates in Fewer Attemptscited by 9
  9. 2025Adversarial Alignment for LLMs Requires Simpler, Reproducible, and More Measurable Objectivescited by 8
  10. 2025Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMscited by 8
  11. 2025The TIP of the Iceberg: Revealing a Hidden Class of Task-in-Prompt Adversarial Attacks on LLMscited by 7
  12. 2024Recent Advances in Attack and Defense Approaches of Large Language Modelscited by 5
  13. 2024Stochastic Monkeys at Play: Random Augmentations Cheaply Break LLM Safety Alignmentcited by 5
  14. 2024Jailbreaking LLM-Controlled Robotscited by 4
  15. 2024Plentiful Jailbreaks with String Compositionscited by 4
  16. 2025Efficient Safety Retrofitting Against Jailbreaking for LLMscited by 4
  17. 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attackscited by 4
  18. 2025TRIDENT: Enhancing Large Language Model Safety with Tri-Dimensional Diversified Red-Teaming Data Synthesiscited by 4
  19. 2025Safety Instincts: LLMs Learn to Trust Their Internal Compass for Self-Defensecited by 4
  20. 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
  21. 2024Harnessing Task Overload for Scalable Jailbreak Attacks on Large Language Modelscited by 3
  22. 2025Semantic Representation Attack against Aligned Large Language Modelscited by 3
  23. 2025RADAR: A Risk-Aware Dynamic Multi-Agent Framework for LLM Safety Evaluation via Role-Specialized Collaborationcited by 3
  24. 2026Controlling Output Rankings in Generative Engines for LLM-based Searchcited by 3
  25. 2025Revealing the Intrinsic Ethical Vulnerability of Aligned Large Language Modelscited by 2
  26. 2025The Structural Safety Generalization Problemcited by 2
  27. 2025InfoFlood: Jailbreaking Large Language Models with Information Overloadcited by 2
  28. 2025Surfacing Subtle Stereotypes: A Multilingual, Debate-Oriented Evaluation of Modern LLMscited by 2
  29. 2025Efficient LLM Safety Evaluation through Multi-Agent Debatecited by 2
  30. 2026Position: Capability Control Should be a Separate Goal From Alignmentcited by 2
  31. 2026Jailbreak Foundry: From Papers to Runnable Attacks for Reproducible Benchmarkingcited by 2
  32. 2026Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamicscited by 2
  33. 2025HauntAttack: When Attack Follows Reasoning as a Shadowcited by 1
  34. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  35. 2025SafetyFlow: An Agent-Flow System for Automated LLM Safety Benchmarkingcited by 1
  36. 2025Bidirectional Intention Inference Enhances LLMs' Defense Against Multi-Turn Jailbreak Attackscited by 1
  37. 2025ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attackcited by 1
  38. 2026Automated Safety Benchmarking: A Multi-agent Pipeline for LVLMscited by 1

50 most cited of 74 All 74 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026LLMs Can Unlearn Refusal with Only 1,000 Benign Samples
  2. 2024Don't Say No: Jailbreaking LLM by Suppressing Refusal
  3. 2026What Do Safety-Aligned LLMs Learn From Mixed Compliance Demonstrations?
  4. 2024Self and Cross-Model Distillation for LLMs: Effective Methods for Refusal Pattern Alignment
  5. 2026Latent-space Attacks for Refusal Evasion in Language Models
  6. 2026Addressing Over-Refusal in LLMs with Competing Rewards
  7. 2024Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes
  8. 2025Understanding Refusal in Language Models with Sparse Autoencoders