# Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes

arXiv: https://arxiv.org/abs/2403.00867 · HTML: https://mlsanity.com/paper/2403.00867/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 30 of its 35 references that are in the map.
- First posted on arXiv 1 Mar 2024.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 30 of 30)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Jailbroken: How Does LLM Safety Training Fail?](https://mlsanity.com/paper/2307.02483/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [Certified Adversarial Robustness via Randomized Smoothing](https://mlsanity.com/paper/1902.02918/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://mlsanity.com/paper/2312.02119/) (2023)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [On Adaptive Attacks to Adversarial Example Defenses](https://mlsanity.com/paper/2002.08347/) (2020)
- [Black-box Adversarial Attacks with Limited Queries and Information](https://mlsanity.com/paper/1804.08598/) (2018)
- [SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks](https://mlsanity.com/paper/2310.03684/) (2023)
- [Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations](https://mlsanity.com/paper/2310.06387/) (2023)
- [Low-Resource Languages Jailbreak GPT-4](https://mlsanity.com/paper/2310.02446/) (2023)
- [Certifying LLM Safety against Adversarial Prompting](https://mlsanity.com/paper/2309.02705/) (2023)
- [SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding](https://mlsanity.com/paper/2402.08983/) (2024)
- [A Primer on Zeroth-Order Optimization in Signal Processing and Machine Learning](https://mlsanity.com/paper/2006.06224/) (2020)
- [Query-Efficient Hard-label Black-box Attack:An Optimization-based Approach](https://mlsanity.com/paper/1807.04457/) (2018)
- [A Theoretical and Empirical Comparison of Gradient Approximations in Derivative-Free Optimization](https://mlsanity.com/paper/1905.01332/) (2019)
- [The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness](https://mlsanity.com/paper/2401.00287/) (2023)
- [In conversation with Artificial Intelligence: aligning language models with human values](https://mlsanity.com/paper/2209.00731/) (2022)

## Built on by (most cited first, 30 of 74)

- [Jailbreak Attacks and Defenses Against Large Language Models: A Survey](https://mlsanity.com/paper/2407.04295/) (2024)
- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts](https://mlsanity.com/paper/2410.10700/) (2024)
- [Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses](https://mlsanity.com/paper/2406.01288/) (2024)
- [JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Models](https://mlsanity.com/paper/2407.01599/) (2024)
- [Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training](https://mlsanity.com/paper/2407.09121/) (2024)
- [SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner](https://mlsanity.com/paper/2406.05498/) (2024)
- [JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulation](https://mlsanity.com/paper/2502.07557/) (2025)
- [Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMs](https://mlsanity.com/paper/2406.09324/) (2024)
- [Adversarial Tuning: Defending Against Jailbreak Attacks for LLMs](https://mlsanity.com/paper/2406.06622/) (2024)
- [Mission Impossible: A Statistical Perspective on Jailbreaking LLMs](https://mlsanity.com/paper/2408.01420/) (2024)
- [SoK: Evaluating Jailbreak Guardrails for Large Language Models](https://mlsanity.com/paper/2506.10597/) (2025)
- [X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usability](https://mlsanity.com/paper/2502.09990/) (2025)
- [VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wild](https://mlsanity.com/paper/2410.00296/) (2024)
- [AutoJailbreak: Exploring Jailbreak Attacks and Defenses through a Dependency Lens](https://mlsanity.com/paper/2406.03805/) (2024)
- [LLMScan: Causal Scan for LLM Misbehavior Detection](https://mlsanity.com/paper/2410.16638/) (2024)
- [Adversarial Representation Engineering: A General Model Editing Framework for Large Language Models](https://mlsanity.com/paper/2404.13752/) (2024)
- [Think Twice, Generate Once: Safeguarding by Progressive Self-Reflection](https://mlsanity.com/paper/2510.01270/) (2025)
- [Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations](https://mlsanity.com/paper/2403.06009/) (2024)
- [PrimeGuard: Safe and Helpful LLMs through Tuning-Free Routing](https://mlsanity.com/paper/2407.16318/) (2024)
- [Jailbreak Attack Initializations as Extractors of Compliance Directions](https://mlsanity.com/paper/2502.09755/) (2025)
- [Self-Destructive Language Model](https://mlsanity.com/paper/2505.12186/) (2025)
- [Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs](https://mlsanity.com/paper/2510.02833/) (2025)
- [Decoupled Alignment for Robust Plug-and-Play Adaptation](https://mlsanity.com/paper/2406.01514/) (2024)
- [A Survey on Responsible Generative AI: What to Generate and What Not](https://mlsanity.com/paper/2404.05783/) (2024)
- [Token Highlighter: Inspecting and Mitigating Jailbreak Prompts for Large Language Models](https://mlsanity.com/paper/2412.18171/) (2024)
- [TeleAI-Safety: A comprehensive LLM jailbreaking benchmark towards attacks, defenses, and evaluations](https://mlsanity.com/paper/2512.05485/) (2025)
- [Knowledge-Driven Multi-Turn Jailbreaking on Large Language Models](https://mlsanity.com/paper/2601.05445/) (2026)
- [Exposing the Ghost in the Transformer: Abnormal Detection for Large Language Models via Hidden State Forensics](https://mlsanity.com/paper/2504.00446/) (2025)
- [CARE: Decoding Time Safety Alignment via Rollback and Introspection Intervention](https://mlsanity.com/paper/2509.06982/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) are becoming a prominent generative AI tool, where the user enters a query and the LLM generates an answer. To reduce harm and misuse, efforts have been made to align these LLMs to human values using advanced training techniques such as Reinforcement Learning from Human Feedback (RLHF). However, recent studies have highlighted the vulnerability of LLMs to adversarial jailbreak attempts aiming at subverting the embedded safety guardrails. To address this challenge, this paper defines and investigates the Refusal Loss of LLMs and then proposes a method called Gradient Cuff to detect jailbreak attempts. Gradient Cuff exploits the unique properties observed in the refusal loss landscape, including functional values and its smoothness, to design an effective two-step detection strategy. Experimental results on two aligned LLMs (LLaMA-2-7B-Chat and Vicuna-7B-V1.5) and six types of jailbreak attacks (GCG, AutoDAN, PAIR, TAP, Base64, and LRL) show that Gradient Cuff can significantly improve the LLM's rejection capability for malicious jailbreak queries, while maintaining the model's performance for benign user queries by adjusting the detection threshold.