mlsanityData to 7 Oct 2026

ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding

arXiv 2402.11889 ↗First posted 19 Feb 2024cs.CLcited by 32

Not on the map. Why?

Abstract (arXiv)

With the development of instruction-tuned large language models (LLMs), improving the safety of LLMs has become more critical. However, the current approaches for aligning the LLMs output with expected safety usually require substantial training efforts, e.g., high-quality safety data and expensive computational resources, which are costly and inefficient. To this end, we present reverse prompt contrastive decoding (ROSE), a simple-yet-effective method to directly boost the safety of existing instruction-tuned LLMs without any additional training. The principle of ROSE is to improve the probability of desired safe output via suppressing the undesired output induced by the carefully-designed reverse prompts. Experiments on 6 safety and 2 general-purpose tasks show that, our ROSE not only brings consistent and significant safety improvements (up to +13.8% safety score) upon 5 types of instruction-tuned LLMs, but also benefits the general-purpose ability of LLMs. In-depth analyses explore the underlying mechanism of ROSE, and reveal when and where to use it.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 31 of its 49 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

31 of 49 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2023Qwen Technical Reportcited by 3,273
  9. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  10. 2018Hierarchical Neural Story Generationcited by 1,182
  11. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
  12. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
19 more
  1. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  2. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  3. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  4. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  5. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  6. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  7. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  8. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  9. 2023Efficient and Effective Text Encoding for Chinese LLaMA and Alpacacited by 302
  10. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  11. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
  12. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  13. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  14. 2023The Capacity for Moral Self-Correction in Large Language Modelscited by 171
  15. 2023Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERTcited by 155
  16. 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
  17. 2023Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Modelscited by 87
  18. 2023CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibilitycited by 80
  19. 2023Mitigating Hallucinations and Off-target Machine Translation with Source-Contrastive and Language-Contrastive Decodingcited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 14; 2026: 5.

32 papers.

Sort:

By area:

  1. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  2. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 100
  3. 2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defendercited by 37
  4. 2024No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attackscited by 28
  5. 2025Beware of Your Po! Measuring and Mitigating AI Safety Risks in Role-Play Fine-Tuning of LLMscited by 22
  6. 2024Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patchingcited by 16
  7. 2025On Almost Surely Safe Alignment of Large Language Models at Inference-Timecited by 13
  8. 2024Unlocking Decoding-time Controllability: Gradient-Free Multi-Objective Alignment with Contrastive Promptscited by 9
  9. 2024Diversity Helps Jailbreak Large Language Modelscited by 9
  10. 2024Safety Alignment of Large Language Models via Contrasting Safe and Harmful Distributionscited by 8
  11. 2025Privacy-Aware Decoding: Mitigating Privacy Leakage of Large Language Models in Retrieval-Augmented Generationcited by 8
  12. 2024mathbbUSCD: Improving Code Generation of LLMs by Uncertainty-Aware Selective Contrastive Decodingcited by 6
20 more
  1. 2024ReflectDiffu:Reflect between Emotion-intent Contagion and Mimicry for Empathetic Response Generation via a RL-Diffusion Frameworkcited by 6
  2. 2024Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspectivecited by 5
  3. 2024Recent Advances in Attack and Defense Approaches of Large Language Modelscited by 5
  4. 2025CARE: Decoding Time Safety Alignment via Rollback and Introspection Interventioncited by 5
  5. 2025Enhancing Input-Label Mapping in In-Context Learning with Contrastive Decodingcited by 4
  6. 2024Building Accurate Translation-Tailored LLMs with Language Aware Instruction Tuningcited by 3
  7. 2025Evolutionary Guided Decoding: Iterative Value Refinement for LLMscited by 3
  8. 2025DP-GTR: Differentially Private Prompt Protection via Group Text Rewritingcited by 2
  9. 2025Chinese Toxic Language Mitigation via Sentiment Polarity Consistent Rewritescited by 2
  10. 2024On the Transformations across Reward Model, Parameter Update, and In-Context Promptcited by 1
  11. 2024What's New in My Data? Novelty Exploration via Contrastive Generationcited by 1
  12. 2025FALCON: Fine-grained Activation Manipulation by Contrastive Orthogonal Unalignment for Large Language Modelcited by 1
  13. 2025LaPA2: Length-Aware Prefix and Prompt Attention Augmentation for Long-Form Controllable Text Generationcited by 1
  14. 2025Exploring and Mitigating Fawning Hallucinations in Large Language Modelscited by 1
  15. 2026The Geometry of Refusal: Linear Instability in Safety-Aligned LLMscited by 1
  16. 2025Reasoning-Aware Multimodal Fusion for Hateful Video Detectioncited by 0
  17. 2026A Systematic Study of Training-Free Methods for Trustworthy Large Language Modelscited by 0
  18. 2026Please refuse to answer me! Mitigating Over-Refusal in Large Language Models via Adaptive Contrastive Decodingcited by 0
  19. 2026Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusioncited by 0
  20. 2026Beyond Tokens: A Survey on Decoding Methods for Large Language and Vision-Language Modelscited by 0