mlsanityData to 7 Oct 2026

The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1

arXiv 2502.12659 ↗First posted 18 Feb 2025cs.CY cs.AIcited by 89

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

The rapid development of large reasoning models (LRMs), such as OpenAI-o3 and DeepSeek-R1, has led to significant improvements in complex reasoning over non-reasoning large language models~(LLMs). However, their enhanced capabilities, combined with the open-source access of models like DeepSeek-R1, raise serious safety concerns, particularly regarding their potential for misuse. In this work, we present a comprehensive safety assessment of these reasoning models, leveraging established safety benchmarks to evaluate their compliance with safety regulations. Furthermore, we investigate their susceptibility to adversarial attacks, such as jailbreaking and prompt injection, to assess their robustness in real-world applications. Through our multi-faceted analysis, we uncover four key findings: (1) There is a significant safety gap between the open-source reasoning models and the o3-mini model, on both safety benchmark and attack, suggesting more safety effort on open LRMs is needed. (2) The stronger the model's reasoning ability, the greater the potential harm it may cause when answering unsafe questions. (3) Safety thinking emerges in the reasoning process of LRMs, but fails frequently against adversarial attacks. (4) The thinking process in R1 models poses greater safety concerns than their final answers. Our study provides insights into the security implications of reasoning models and highlights the need for further advancements in R1 models' safety to close the gap.

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 20 of its 33 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

20 of 33 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Deliberative Alignment: Reasoning Enables Safer Language Models20 Dec

Other references · 19

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  3. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  4. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  5. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 476
  6. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  7. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 406
  8. 2024Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Expertscited by 369
  9. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 341
  10. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  11. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 224
  12. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
7 more
  1. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 176
  2. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 160
  3. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 145
  4. 2024What is in Your Safe Data? Identifying Benign Data that Breaks Safetycited by 114
  5. 2024Rule Based Rewards for Language Model Safetycited by 106
  6. 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 69
  7. 2024Quantile Regression for Distributional Reward Models in RLHFcited by 59

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 65; 2026: 24.

89 papers.

Sort:

By area:

  1. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  2. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  3. 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonablecited by 120
  4. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Datacited by 67
  5. 2025Effectively Controlling Reasoning Models through Thinking Interventioncited by 63
  6. 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capabilitycited by 58
  7. 2025Safety in Large Reasoning Models: A Surveycited by 58
  8. 2025FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoningcited by 53
  9. 2025Assessing Judging Bias in Large Reasoning Models: An Empirical Studycited by 30
  10. 2025SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Modelscited by 29
  11. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 29
  12. 2025Inference-Time Hyper-Scaling with KV Cache Compressioncited by 28
38 more
  1. 2025Leaky Thoughts: Large Reasoning Models Are Not Private Thinkerscited by 25
  2. 2025Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findingscited by 24
  3. 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignmentcited by 24
  4. 2025Reasoning as an Adaptive Defense for Safetycited by 22
  5. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  6. 2025Trade-offs in Large Reasoning Models: An Empirical Analysis of Deliberative and Adaptive Reasoning over Foundational Capabilitiescited by 20
  7. 2025SafeKey: Amplifying Aha-Moment Insights for Safety Reasoningcited by 20
  8. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 19
  9. 2025Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Modelcited by 18
  10. 2025How Should We Enhance the Safety of Large Reasoning Models: An Empirical Studycited by 18
  11. 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasetscited by 18
  12. 2025Alignment and Safety in Large Language Models: Safety Mechanisms, Training Paradigms, and Emerging Challengescited by 14
  13. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 13
  14. 2025Large Reasoning Models Learn Better Alignment from Flawed Thinkingcited by 13
  15. 2025Chain-of-Thought Hijackingcited by 13
  16. 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skillscited by 12
  17. 2025VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Modelscited by 11
  18. 2025Sherlock: Self-Correcting Reasoning in Vision-Language Modelscited by 11
  19. 2025Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewardscited by 11
  20. 2025Towards Safe Reasoning in Large Reasoning Models via Corrective Interventioncited by 11
  21. 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?cited by 11
  22. 2025Process or Result? Manipulated Ending Tokens Can Mislead Reasoning LLMs to Ignore the Correct Reasoning Stepscited by 9
  23. 2025ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Modelscited by 9
  24. 2025Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Trainingcited by 9
  25. 2026Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?cited by 8
  26. 2025Practical Reasoning Interruption Attacks on Reasoning Large Language Modelscited by 7
  27. 2025UnsafeChain: Enhancing Reasoning Model Safety via Hard Casescited by 7
  28. 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Modelscited by 7
  29. 2025LoRA is All You Need for Safety Alignment of Reasoning LLMscited by 6
  30. 2025R1-ACT: Efficient Reasoning Model Safety Alignment by Activating Safety Knowledgecited by 6
  31. 2026What Matters For Safety Alignment?cited by 6
  32. 2026PrivAct: Internalizing Contextual Privacy Preservation via Multi-Agent Preference Trainingcited by 6
  33. 2025Path Drift in Large Reasoning Models:How First-Person Commitments Override Safetycited by 5
  34. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Modelscited by 5
  35. 2025Reasoning Up the Instruction Ladder for Controllable Language Modelscited by 5
  36. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 5
  37. 2025AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Modelscited by 4
  38. 2025SafeRBench: Dissecting the Reasoning Safety of Large Language Modelscited by 4

50 most cited of 89 All 89 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities
  2. 2026Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Models
  3. 2024Large Language Model Safety: A Holistic Survey
  4. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety
  5. 2026On the Efficiency-Safety Dilemma in Large Reasoning Models
  6. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Models
  7. 2026SFCoT: Safer Chain-of-Thought via Active Safety Evaluation and Calibration
  8. 2025Red Teaming Large Reasoning Models