mlsanityData to 14 Oct 2026

Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable

arXiv 2503.00555 ↗First posted 1 Mar 2025cs.CR cs.AI cs.LGcited by 122

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Safety alignment is an important procedure before the official deployment of a Large Language Model (LLM). While safety alignment has been extensively studied for LLM, there is still a large research gap for Large Reasoning Models (LRMs) that equip with improved reasoning capability. We in this paper systematically examine a simplified pipeline for producing safety aligned LRMs. With our evaluation of various LRMs, we deliver two main findings: i) Safety alignment can be done upon the LRM to restore its safety capability. ii) Safety alignment leads to a degradation of the reasoning capability of LRMs. The two findings show that there exists a trade-off between reasoning and safety capability with the sequential LRM production pipeline. The discovered trade-off, which we name Safety Tax, should shed light on future endeavors of safety research on LRMs. As a by-product, we curate a dataset called DirectRefusal, which might serve as an alternative dataset for safety alignment. Our source code is available at https://github.com/git-disl/Safety-Tax.

Key facts. Cited by 122 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 32 of its 40 references that are in the map. First posted on arXiv 1 Mar 2025.

Builds on

32 of 40 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R118 Feb
  2. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities17 Feb
  3. 2025To Think or Not to Think: Exploring the Unthinking Vulnerability in Large Reasoning Models16 Feb
  4. 2025Are Smarter LLMs Safer? Exploring Safety-Reasoning Trade-offs in Prompting and Fine-Tuning13 Feb
  5. 2025The dark deep side of DeepSeek: Fine-tuning attacks against the safety alignment of CoT-enabled models3 Feb

Other references · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  7. 2023Let's Verify Step by Stepcited by 3,789
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  9. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  10. 2025s1: Simple test-time scalingcited by 1,341
  11. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  12. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
15 more
  1. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  2. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  3. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
  4. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  5. 2025LIMO: Less is More for Reasoningcited by 498
  6. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 463
  7. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
  8. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruningcited by 279
  9. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 241
  10. 2019Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computingcited by 228
  11. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuningcited by 169
  12. 2024Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Surveycited by 90
  13. 2023Pairwise Proximal Policy Optimization: Harnessing Relative Feedback for LLM Alignmentcited by 57
  14. 2025Mix Data or Merge Models? Balancing the Helpfulness, Honesty, and Harmlessness of Large Language Model via Model Mergingcited by 23
  15. 2024H3Fusion: Helpful, Harmless, Honest Fusion of Aligned LLMscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 56; 2026: 66.

122 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  2. 2025Effectively Controlling Reasoning Models through Thinking Interventioncited by 65
  3. 2025Safety in Large Reasoning Models: A Surveycited by 64
  4. 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capabilitycited by 59
  5. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 42
  6. 2025Assessing Judging Bias in Large Reasoning Models: An Empirical Studycited by 30
  7. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 30
  8. 2025SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Modelscited by 29
  9. 2025Leaky Thoughts: Large Reasoning Models Are Not Private Thinkerscited by 28
  10. 2025When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoningcited by 26
  11. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  12. 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignmentcited by 24
38 more
  1. 2025Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilitiescited by 20
  2. 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasetscited by 20
  3. 2025Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Modelscited by 19
  4. 2025How Should We Enhance the Safety of Large Reasoning Models: An Empirical Studycited by 18
  5. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  6. 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skillscited by 14
  7. 2025Can AI Perceive Physical Danger and Intervene?cited by 14
  8. 2025Large Reasoning Models Learn Better Alignment from Flawed Thinkingcited by 13
  9. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  10. 2025Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewardscited by 12
  11. 2025DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Modelscited by 12
  12. 2025SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimizationcited by 12
  13. 2025Towards Safe Reasoning in Large Reasoning Models via Corrective Interventioncited by 11
  14. 2025Mitigating the Safety Alignment Tax with Null-Space Constrained Policy Optimizationcited by 11
  15. 2026Reducing the Safety Tax in LLM Safety Alignment with On-Policy Self-Distillationcited by 11
  16. 2026THINKSAFE: Self-Generated Safety Alignment for Reasoning Modelscited by 10
  17. 2025ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Modelscited by 9
  18. 2025SafeSearch: Do Not Trade Safety for Utility in LLM Search Agentscited by 9
  19. 2025ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoningcited by 8
  20. 2026Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studiescited by 8
  21. 2025R1dacted: Investigating Local Censorship in DeepSeek's R1 Language Modelcited by 7
  22. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Modelscited by 7
  23. 2025UnsafeChain: Enhancing Reasoning Model Safety via Hard Casescited by 7
  24. 2025OmniGuard: Unified Omni-Modal Guardrails with Deliberate Reasoningcited by 7
  25. 2025LoRA is All You Need for Safety Alignment of Reasoning LLMscited by 6
  26. 2025R1-ACT: Efficient Reasoning Model Safety Alignment by Activating Safety Knowledgecited by 6
  27. 2025Read the Scene, Not the Script: Outcome-Aware Safety for LLMscited by 6
  28. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Modelscited by 6
  29. 2026TriPlay-RL: Tri-Role Self-Play Reinforcement Learning for LLM Safety Alignmentcited by 6
  30. 2026Training Agents to Self-Report Misbehaviorcited by 6
  31. 2025Murphys Laws of AI Alignment: Why the Gap Always Winscited by 5
  32. 2025Interpretable Safety Alignment via SAE-Constructed Low-Rank Subspace Adaptationcited by 5
  33. 2025IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinementcited by 4
  34. 2025Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberationcited by 4
  35. 2025Knowledge-Level Consistency Reinforcement Learning: Dual-Fact Alignment for Long-Form Factualitycited by 4
  36. 2025GuardTrace-VL: Detecting Unsafe Multimodel Reasoning via Iterative Safety Supervisioncited by 4
  37. 2025ProGuard: Towards Proactive Multimodal Safeguardcited by 4
  38. 2026SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignmentcited by 4

50 most cited of 122 All 122 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Superficial Safety Alignment Hypothesis
  2. 2024Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patching
  3. 2025SaRO: Enhancing LLM Safety through Reasoning-based Alignment
  4. 2026Beyond Safe Data: Pretraining-Stage Alignment with Regular Safety Reflection
  5. 2025SafeLawBench: Towards Safe Alignment of Large Language Models
  6. 2026Towards Safer Large Reasoning Models by Promoting Safety Decision-Making before Chain-of-Thought Generation
  7. 2024Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements
  8. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Data