mlsanityData to 14 Oct 2026

Safety in Large Reasoning Models: A Survey

arXiv 2504.17704 ↗First posted 24 Apr 2025cs.CLcited by 64

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

Large Reasoning Models (LRMs) have exhibited extraordinary prowess in tasks like mathematics and coding, leveraging their advanced reasoning capabilities. Nevertheless, as these capabilities progress, significant concerns regarding their vulnerabilities and safety have arisen, which can pose challenges to their deployment and application in real-world settings. This paper presents a comprehensive survey of LRMs, meticulously exploring and summarizing the newly emerged safety risks, attacks, and defense strategies. By organizing these elements into a detailed taxonomy, this work aims to offer a clear and structured understanding of the current safety landscape of LRMs, facilitating future research and development to enhance the security and reliability of these powerful models.

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 88 of its 118 references that are in the map. First posted on arXiv 24 Apr 2025.

Builds on

88 of 118 references.

Closest ancestors · 26

Same direction, newest first

  1. 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capability14 Apr
  2. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems12 Apr
  3. 2025SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models9 Apr
  4. 2025ShadowCoT: Cognitive Hijacking for Stealthy Reasoning Backdoors in LLMs8 Apr
  5. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Data2 Apr
  6. 2025Process or Result? Manipulated Ending Tokens Can Mislead Reasoning LLMs to Ignore the Correct Reasoning Steps25 Mar
  7. 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable1 Mar
  8. 2025A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos19 Feb
  9. 2025ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrails19 Feb
  10. 2025Demonstrating specification gaming in reasoning models18 Feb
  11. 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking18 Feb
  12. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking18 Feb
14 more
  1. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R118 Feb
  2. 2025Nuclear Deployed: Analyzing Catastrophic Risks in Decision-making of Autonomous LLM Agents17 Feb
  3. 2025To Think or Not to Think: Exploring the Unthinking Vulnerability in Large Reasoning Models16 Feb
  4. 2025Safety Evaluation of DeepSeek Models in Chinese Contexts16 Feb
  5. 2025STAIR: Improving Safety Alignment with Introspective Reasoning4 Feb
  6. 2025Trading Inference-Time Compute for Adversarial Robustness31 Jan
  7. 2025o3-mini vs DeepSeek-R1: Which One is Safer?30 Jan
  8. 2025Early External Safety Testing of OpenAI's o3-mini: Insights from the Pre-Deployment Evaluation29 Jan
  9. 2024Deliberative Alignment: Reasoning Enables Safer Language Models20 Dec
  10. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment27 Nov
  11. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code12 Mar
  12. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models20 Jan
  13. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  14. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May

Other references · 62

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  5. 2023Let's Verify Step by Stepcited by 3,789
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  8. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,055
  9. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  10. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  11. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
  12. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
38 more
  1. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
  2. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 370
  3. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  4. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 295
  5. 2021Recursively Summarizing Books with Human Feedbackcited by 290
  6. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 289
  7. 2023Large Language Model Alignment: A Surveycited by 243
  8. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
  9. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 239
  10. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 231
  11. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
  12. 2021Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Triggercited by 222
  13. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 214
  14. 2024Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understandingcited by 188
  15. 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 183
  16. 2021Challenges in Detoxifying Language Modelscited by 169
  17. 2025The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Taskscited by 152
  18. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 151
  19. 2024AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Expertscited by 136
  20. 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrailscited by 132
  21. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 132
  22. 2024Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversationscited by 129
  23. 2023Black-Box Prompt Optimization: Aligning Large Language Models without Model Trainingcited by 122
  24. 2020Sponge Examples: Energy-Latency Attacks on Neural Networkscited by 111
  25. 2021Challenges in Automated Debiasing for Toxic Language Detectioncited by 96
  26. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 86
  27. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Modelcited by 85
  28. 2024InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidancecited by 83
  29. 2024Does Refusal Training in LLMs Generalize to the Past Tense?cited by 79
  30. 2022Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Modelscited by 68
  31. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 67
  32. 2023CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generationcited by 66
  33. 2025Effectively Controlling Reasoning Models through Thinking Interventioncited by 65
  34. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 61
  35. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 60
  36. 2024Large Language Model Safety: A Holistic Surveycited by 59
  37. 2024Multi-Turn Context Jailbreak Attack on Large Language Models From First Principlescited by 56
  38. 2025Can Indirect Prompt Injection Attacks Be Detected and Removed?cited by 52

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 40; 2026: 24.

64 papers.

Sort:

By area:

  1. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 42
  2. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 38
  3. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 37
  4. 2025EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systemscited by 37
  5. 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
  6. 2025Leaky Thoughts: Large Reasoning Models Are Not Private Thinkerscited by 28
  7. 2025Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injectioncited by 25
  8. 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignmentcited by 24
  9. 2025Why LLM Safety Guardrails Collapse After Fine-tuning: A Similarity Analysis Between Alignment and Fine-tuning Datasetscited by 20
  10. 2025False Sense of Security: Why Probing-based Malicious Input Detection Fails to Generalizecited by 19
  11. 2025Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instructioncited by 16
  12. 2025SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agentscited by 15
38 more
  1. 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skillscited by 14
  2. 2025TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewardscited by 14
  3. 2025Large Reasoning Models Learn Better Alignment from Flawed Thinkingcited by 13
  4. 2025Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025cited by 12
  5. 2025Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Trainingcited by 12
  6. 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?cited by 11
  7. 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 11
  8. 2025SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinationscited by 9
  9. 2025BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Modelscited by 8
  10. 2026EvoClinician: A Self-Evolving Agent for Multi-Turn Medical Diagnosis via Test-Time Evolutionary Learningcited by 8
  11. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Modelscited by 7
  12. 2025Efficient Reasoning via Chain of Unconscious Thoughtcited by 7
  13. 2025UnsafeChain: Enhancing Reasoning Model Safety via Hard Casescited by 7
  14. 2026What Matters For Safety Alignment?cited by 7
  15. 2026EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Modelscited by 5
  16. 2025Thought Purity: A Defense Framework For Chain-of-Thought Attackcited by 4
  17. 2025ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Momentscited by 4
  18. 2025AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Modelscited by 4
  19. 2025Backdoor-Powered Prompt Injection Attacks Nullify Defense Methodscited by 3
  20. 2026How Does the Thinking Step Influence Model Safety? An Entropy-based Safety Reminder for LRMscited by 3
  21. 2026Backdoors in RLVR: Jailbreak Backdoors in LLMs From Verifiable Rewardcited by 3
  22. 2026Do Thinking Tokens Help with Safety?cited by 3
  23. 2025Does More Inference-Time Compute Really Help Robustness?cited by 2
  24. 2025Rethinking Reasoning: A Survey on Reasoning-based Backdoors in LLMscited by 2
  25. 2025Red Teaming Large Reasoning Modelscited by 2
  26. 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Abilitycited by 2
  27. 2026When Reasoning Leaks Membership: Membership Inference Attack on Black-box Large Reasoning Modelscited by 2
  28. 2026Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Modelscited by 2
  29. 2026Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representationscited by 2
  30. 2026Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Modelscited by 2
  31. 2025PD3F: A Pluggable and Dynamic DoS-Defense Framework Against Resource Consumption Attacks Targeting Large Language Modelscited by 1
  32. 2025HauntAttack: When Attack Follows Reasoning as a Shadowcited by 1
  33. 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1
  34. 2026Steerability of Instrumental-Convergence Tendencies in LLMscited by 1
  35. 2026GAMBIT: A Gamified Jailbreak Framework for Multimodal Large Language Modelscited by 1
  36. 2026Reasoning Hijacking: The Fragility of Reasoning Alignment in Large Language Modelscited by 1
  37. 2026RAPO: Risk-Aware Preference Optimization for Generalizable Safe Reasoningcited by 1
  38. 2026ReasoningLens: Hierarchical Visualization and Diagnostic Auditing for Large Reasoning Modelscited by 1

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025How Should We Enhance the Safety of Large Reasoning Models: An Empirical Study
  2. 2025SafeRBench: Dissecting the Reasoning Safety of Large Language Models
  3. 2026Towards Safer Large Reasoning Models by Promoting Safety Decision-Making before Chain-of-Thought Generation
  4. 2025Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Model
  5. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models
  6. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities
  7. 2025R1-ACT: Efficient Reasoning Model Safety Alignment by Activating Safety Knowledge
  8. 2025Safety Through Reasoning: An Empirical Study of Reasoning Guardrail Models