mlsanityData to 7 Oct 2026

Reasoning as an Adaptive Defense for Safety

arXiv 2507.00971 ↗First posted 1 Jul 2025cs.LG cs.AIcited by 22

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Reasoning methods that adaptively allocate test-time compute have advanced LLM performance on easy to verify domains such as math and code. In this work, we study how to utilize this approach to train models that exhibit a degree of robustness to safety vulnerabilities, and show that doing so can provide benefits. We build a recipe called textitTARS (Training Adaptive Reasoners for Safety), a reinforcement learning (RL) approach that trains models to reason about safety using chain-of-thought traces and a reward signal that balances safety with task completion. To build TARS, we identify three critical design choices: (1) a ``lightweight'' warmstart SFT stage, (2) a mix of harmful, harmless, and ambiguous prompts to prevent shortcut behaviors such as too many refusals, and (3) a reward function to prevent degeneration of reasoning capabilities during training. Models trained with TARS exhibit adaptive behaviors by spending more compute on ambiguous queries, leading to better safety-refusal trade-offs. They also internally learn to better distinguish between safe and unsafe prompts and attain greater robustness to both white-box (e.g., GCG) and black-box attacks (e.g., PAIR). Overall, our work provides an effective, open recipe for training LLMs against jailbreaks and harmful requests by reasoning per prompt.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 67 of its 74 references that are in the map. First posted on arXiv 1 Jul 2025.

Builds on

67 of 74 references.

Closest ancestors · 16

Same direction, newest first

  1. 2025Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Ciphers22 May
  2. 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capability14 Apr
  3. 2025SaRO: Enhancing LLM Safety through Reasoning-based Alignment13 Apr
  4. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Data2 Apr
  5. 2025Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behavior22 Mar
  6. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety6 Mar
  7. 2025A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos19 Feb
  8. 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking18 Feb
  9. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking18 Feb
  10. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R118 Feb
  11. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities17 Feb
  12. 2025STAIR: Improving Safety Alignment with Introspective Reasoning4 Feb
4 more
  1. 2025Trading Inference-Time Compute for Adversarial Robustness31 Jan
  2. 2024Deliberative Alignment: Reasoning Enables Safer Language Models20 Dec
  3. 2024Recursive Introspection: Teaching Language Model Agents How to Self-Improve25 Jul
  4. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan

Other references · 51

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  9. 2023Let's Verify Step by Stepcited by 3,598
  10. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  11. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  12. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
38 more
  1. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  2. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  3. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  4. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  5. 2025s1: Simple test-time scalingcited by 1,311
  6. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  7. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  8. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  9. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  10. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  11. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  12. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  13. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  14. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  15. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  16. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  17. 2024A StrongREJECT for Empty Jailbreakscited by 352
  18. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  19. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 323
  20. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  21. 2024Iterative Reasoning Preference Optimizationcited by 244
  22. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  23. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
  24. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 206
  25. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 189
  26. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  27. 2024Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMscited by 142
  28. 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrailscited by 122
  29. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
  30. 2025OctoThinker: Mid-training Incentivizes Reinforcement Learning Scalingcited by 110
  31. 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoningcited by 91
  32. 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
  33. 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysiscited by 61
  34. 2025e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMscited by 56
  35. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspectivecited by 44
  36. 2024Backtracking Improves Generation Safetycited by 31
  37. 2024Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Modelscited by 23
  38. 2024What Do Learning Dynamics Reveal About Generalization in LLM Reasoning?cited by 22

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 13.

22 papers.

Sort:

By area:

  1. 2026Constitutional Classifiers++: Efficient Production-Grade Defenses against Universal Jailbreakscited by 36
  2. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  3. 2025RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problemscited by 21
  4. 2025Towards Safe Reasoning in Large Reasoning Models via Corrective Interventioncited by 11
  5. 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?cited by 11
  6. 2025CaRT: Teaching LLM Agents to Know When They Know Enoughcited by 5
  7. 2025Reasoning Up the Instruction Ladder for Controllable Language Modelscited by 5
  8. 2026Robust Policy Optimization to Prevent Catastrophic Forgettingcited by 3
  9. 2025The Resurgence of GCG Adversarial Attacks on Large Language Modelscited by 2
  10. 2025Strong but Brittle: Simple Attacks Subvert Reasoning-based Safety Guardrailscited by 2
  11. 2026Mitigating the Safety-utility Trade-off in LLM Alignment via Adaptive Safe Context Learningcited by 2
  12. 2026SafeDream: Safety World Model for Proactive Early Jailbreak Detectioncited by 2
10 more
  1. 2025Learning to Extract Context for Context-Aware LLM Inferencecited by 1
  2. 2026RAPO: Risk-Aware Preference Optimization for Generalizable Safe Reasoningcited by 1
  3. 2026Intent Laundering: AI Safety Datasets Are Not What They Seemcited by 1
  4. 2026Amplification Effects in Test-Time Reinforcement Learning: Safety and Reasoning Vulnerabilitiescited by 1
  5. 2026Self-ReSET: Learning to Self-Recover from Unsafe Reasoning Trajectoriescited by 1
  6. 2026Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Modelscited by 0
  7. 2026Addressing Over-Refusal in LLMs with Competing Rewardscited by 0
  8. 2026EOPSA: Efficient On-Policy Self-Distilled Safety Alignmentcited by 0
  9. 2026DeShortcut-Align: Decoupling Spurious Shortcuts for Robust Safety Alignment in Large Reasoning Modelscited by 0
  10. 2026Towards Mitigating Deceptive Safety Alignment in Large Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SafeWork-R1: Coevolving Safety and Intelligence under the AI-45circ Law
  2. 2026Internalizing Safety Understanding in Large Reasoning Models via Verification
  3. 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Ability
  4. 2026Safety Recovery in Reasoning Models Is Only a Few Early Steering Steps Away
  5. 2026Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models
  6. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models
  7. 2026Chain of Risk: Safety Failures in Large Reasoning Models and Mitigation via Adaptive Multi-Principle Steering
  8. 2025AlphaAlign: Incentivizing Safety Alignment with Extremely Simplified Reinforcement Learning