mlsanityData to 7 Oct 2026

Enhancing Model Defense Against Jailbreaks with Proactive Safety Reasoning

arXiv 2501.19180 ↗First posted 31 Jan 2025cs.CR cs.AIcited by 11

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Large language models (LLMs) are vital for a wide range of applications yet remain susceptible to jailbreak threats, which could lead to the generation of inappropriate responses. Conventional defenses, such as refusal and adversarial training, often fail to cover corner cases or rare domains, leaving LLMs still vulnerable to more sophisticated attacks. We propose a novel defense strategy, Safety Chain-of-Thought (SCoT), which harnesses the enhanced textitreasoning capabilities of LLMs for proactive assessment of harmful inputs, rather than simply blocking them. SCoT augments any refusal training datasets to critically analyze the intent behind each request before generating answers. By employing proactive reasoning, SCoT enhances the generalization of LLMs across varied harmful queries and scenarios not covered in the safety alignment corpus. Additionally, it generates detailed refusals specifying the rules violated. Comparative evaluations show that SCoT significantly surpasses existing defenses, reducing vulnerability to out-of-distribution issues and adversarial manipulations while maintaining strong general capabilities.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 28 of its 64 references that are in the map. First posted on arXiv 31 Jan 2025.

Builds on

28 of 64 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024OpenAI o1 System Card21 Dec
  2. 2024Deliberative Alignment: Reasoning Enables Safer Language Models20 Dec

Other references · 26

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2023Mistral 7Bcited by 3,065
  11. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  12. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
14 more
  1. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  2. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  3. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
  4. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  5. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 220
  6. 2024Simulating Classroom Education with LLM-Empowered Agentscited by 125
  7. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 114
  8. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  9. 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialoguecited by 64
  10. 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neuronscited by 47
  11. 2024MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialoguecited by 28
  12. 2023Learning and Forgetting Unsafe Examples in Large Language Modelscited by 24
  13. 2025LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Languagecited by 17
  14. 2024Multi-round jailbreak attack on large language modelscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.

  1. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  2. 2025How Should We Enhance the Safety of Large Reasoning Models: An Empirical Studycited by 18
  3. 2025AlphaAlign: Incentivizing Safety Alignment with Extremely Simplified Reinforcement Learningcited by 14
  4. 2025CoTGuard: Using Chain-of-Thought Triggering for Copyright Protection in Multi-Agent LLM Systemscited by 3
  5. 2026Backdoors in RLVR: Jailbreak Backdoors in LLMs From Verifiable Rewardcited by 3
  6. 2025Strong but Brittle: Simple Attacks Subvert Reasoning-based Safety Guardrailscited by 2
  7. 2026RvB: Automating AI System Hardening via Iterative Red-Blue Gamescited by 2
  8. 2026Why Agents Compromise Safety Under Pressurecited by 2
  9. 2026SoK: Robustness in Large Language Models against Jailbreak Attackscited by 2
  10. 2026Inverting the Shield: Systematically Generating Safety Tests from Policy Specificationscited by 0
  11. 2026Addressing Over-Refusal in LLMs with Competing Rewardscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking
  2. 2025Reasoned Safety Alignment: Ensuring Jailbreak Defense via Answer-Then-Check
  3. 2025ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning
  4. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety
  5. 2024Reasoning as a Weapon: Adaptive Dual-Path Jailbreak Attack on Large Language Models
  6. 2026SFCoT: Safer Chain-of-Thought via Active Safety Evaluation and Calibration
  7. 2025Proactive defense against LLM Jailbreak
  8. 2025SAID: Safety-Aware Intent Defense via Prefix Probing for Large Language Models