mlsanityData to 14 Oct 2026

How Should We Enhance the Safety of Large Reasoning Models: An Empirical Study

arXiv 2505.15404 ↗First posted 21 May 2025cs.CLcited by 18

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Large Reasoning Models (LRMs) have achieved remarkable success on reasoning-intensive tasks such as mathematics and programming. However, their enhanced reasoning capabilities do not necessarily translate to improved safety performance-and in some cases, may even degrade it. This raises an important research question: how should we enhance the safety of LRMs? In this paper, we present a comprehensive empirical study on how to enhance the safety of LRMs through Supervised Fine-Tuning (SFT). Our investigation begins with an unexpected observation: directly distilling safe responses from DeepSeek-R1 fails to significantly enhance safety. We analyze this phenomenon and identify five key risky patterns that contribute to it. We then demonstrate that explicitly addressing these issues during the data distillation process can lead to substantial safety improvements. Next, we explore whether a long and complex reasoning process is necessary for achieving safety. Interestingly, we find that simply using short or template-based reasoning process can attain comparable safety performance. These findings prompt a deeper reflection on the role of reasoning in ensuring safety. Finally, we conduct a comprehensive ablation study to reveal the impact of different training configurations. Overall, we hope our empirical study could provide a more holistic picture on enhancing the safety of LRMs. The code and data used in our experiments are released in https://github.com/thu-coai/LRM-Safety-Study.

Key facts. Cited by 18 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 28 of its 29 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

28 of 29 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capability14 Apr
  2. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Data2 Apr
  3. 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable1 Mar
  4. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R118 Feb
  5. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities17 Feb
  6. 2025Enhancing Model Defense Against Jailbreaks with Proactive Safety Reasoning31 Jan
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code12 Mar
  8. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan

Other references · 20

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2023Let's Verify Step by Stepcited by 3,789
  6. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  8. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  10. 2025s1: Simple test-time scalingcited by 1,341
  11. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  12. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
8 more
  1. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 469
  2. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  3. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  4. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
  5. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritizationcited by 170
  6. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 157
  7. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editingcited by 79
  8. 2023Ethicist: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimationcited by 36

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 6.

18 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  2. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  3. 2025Large Reasoning Models Learn Better Alignment from Flawed Thinkingcited by 13
  4. 2025Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewardscited by 12
  5. 2025DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Modelscited by 12
  6. 2025Towards Safe Reasoning in Large Reasoning Models via Corrective Interventioncited by 11
  7. 2025R1-ACT: Efficient Reasoning Model Safety Alignment by Activating Safety Knowledgecited by 6
  8. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Modelscited by 6
  9. 2025Towards Understanding the Cognitive Habits of Large Reasoning Modelscited by 4
  10. 2025AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Modelscited by 4
  11. 2026Mitigating Safety Tax via Distribution-Grounded Refinement in Large Reasoning Modelscited by 3
  12. 2025EASE: Practical and Efficient Safety Alignment for Small Language Modelscited by 2
6 more
  1. 2025Red Teaming Large Reasoning Modelscited by 2
  2. 2026Mitigating the Safety-utility Trade-off in LLM Alignment via Adaptive Safe Context Learningcited by 2
  3. 2026Reasoning Structure Matters for Safety Alignment of Reasoning Modelscited by 1
  4. 2026Chain of Risk: Safety Failures in Large Reasoning Models and Mitigation via Adaptive Multi-Principle Steeringcited by 1
  5. 2026Reasoning-targeted Jailbreak Attacks on Large Reasoning Models via Semantic Triggers and Psychological Framingcited by 0
  6. 2026Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SafeRBench: Dissecting the Reasoning Safety of Large Language Models
  2. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Models
  3. 2026Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models
  4. 2025Safety in Large Reasoning Models: A Survey
  5. 2026Towards Safer Large Reasoning Models by Promoting Safety Decision-Making before Chain-of-Thought Generation
  6. 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Ability
  7. 2025Safety Through Reasoning: An Empirical Study of Reasoning Guardrail Models
  8. 2025Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Model