mlsanityData to 14 Oct 2026

SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment

arXiv 2505.14667 ↗First posted 20 May 2025cs.AI cs.CLcited by 24

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Large Reasoning Models (LRMs) have become powerful tools for complex problem solving, but their structured reasoning pathways can lead to unsafe outputs when exposed to harmful prompts. Existing safety alignment methods reduce harmful outputs but can degrade reasoning depth, leading to significant trade-offs in complex, multi-step tasks, and remain vulnerable to sophisticated jailbreak attacks. To address this, we introduce SAFEPATH, a lightweight alignment method that fine-tunes LRMs to emit a short, 8-token Safety Primer at the start of their reasoning, in response to harmful prompts, while leaving the rest of the reasoning process unsupervised. Empirical results across multiple benchmarks indicate that SAFEPATH effectively reduces harmful outputs while maintaining reasoning performance. Specifically, SAFEPATH reduces harmful responses by up to 90.0% and blocks 83.3% of jailbreak attempts in the DeepSeek-R1-Distill-Llama-8B model, while requiring 295.9x less compute than Direct Refusal and 314.1x less than SafeChain. We further introduce a zero-shot variant that requires no fine-tuning. In addition, we provide a comprehensive analysis of how existing methods in LLMs generalize, or fail, when applied to reasoning-centric models, revealing critical gaps and new directions for safer AI.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 58 of its 60 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

58 of 60 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Safety in Large Reasoning Models: A Survey24 Apr
  2. 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capability14 Apr
  3. 2025SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models9 Apr
  4. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Data2 Apr
  5. 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable1 Mar
  6. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R118 Feb
  7. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities17 Feb
  8. 2025GuardReasoner: Towards Reasoning-based LLM Safeguards30 Jan
  9. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models20 Jan

Other references · 49

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  9. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  10. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  11. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  12. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
37 more
  1. 2017Deep reinforcement learning from human preferencescited by 3,912
  2. 2021Program Synthesis with Large Language Modelscited by 3,637
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  7. 2025s1: Simple test-time scalingcited by 1,341
  8. 2022Editing Models with Task Arithmeticcited by 1,120
  9. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  10. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  11. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
  12. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  13. 2024Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearningcited by 504
  14. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 489
  15. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  16. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
  17. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 469
  18. 2024A StrongREJECT for Empty Jailbreakscited by 385
  19. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
  20. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
  21. 2023Low-Resource Languages Jailbreak GPT-4cited by 309
  22. 2025Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Modelscited by 293
  23. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 235
  24. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 223
  25. 2023At Which Training Stage Does Code Data Help LLMs Reasoning?cited by 82
  26. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 80
  27. 2024EasyJailbreak: A Unified Framework for Jailbreaking Large Language Modelscited by 60
  28. 2024Large Language Model Safety: A Holistic Surveycited by 59
  29. 2024Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledgecited by 51
  30. 2023Bypassing the Safety Training of Open-Source LLMs with Priming Attackscited by 50
  31. 2025The First Few Tokens Are All You Need: An Efficient and Effective Unsupervised Prefix Fine-Tuning Method for Reasoning Modelscited by 31
  32. 2025Representation Bending for Large Language Model Safetycited by 30
  33. 2025Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findingscited by 27
  34. 2024Perceive, Reflect, and Plan: Designing LLM Agent for Goal-Directed City Navigation without Instructionscited by 25
  35. 2024Large Language Models Still Exhibit Bias in Long Textcited by 16
  36. 2025Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Controlcited by 14
  37. 2024Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Modelscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 15.

24 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  2. 2025Large Reasoning Models Learn Better Alignment from Flawed Thinkingcited by 13
  3. 2025Distributional AGI Safetycited by 12
  4. 2025R-TOFU: Unlearning in Large Reasoning Modelscited by 9
  5. 2025A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Modelscited by 9
  6. 2026What Matters For Safety Alignment?cited by 7
  7. 2026Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden Statescited by 7
  8. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Modelscited by 6
  9. 2025ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Momentscited by 4
  10. 2025Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberationcited by 4
  11. 2026Safety Recovery in Reasoning Models Is Only a Few Early Steering Steps Awaycited by 3
  12. 2026Do Thinking Tokens Help with Safety?cited by 3
12 more
  1. 2026Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representationscited by 2
  2. 2026Towards Safer Large Reasoning Models by Promoting Safety Decision-Making before Chain-of-Thought Generationcited by 2
  3. 2026Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agentscited by 1
  4. 2025Concept Concentration for Faithful Representation Interventioncited by 0
  5. 2026Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Modelscited by 0
  6. 2026Dual-Adversarial Safety Alignment: Cultivating Intrinsic Threat Comprehension in LRMscited by 0
  7. 2026INTENT-AS-A-TOOL Makes it Easy to Track Agentic Misalignmentcited by 0
  8. 2026Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusalcited by 0
  9. 2026First Token Matters: Understanding Safety Collapse in Large Reasoning Modelscited by 0
  10. 2026Alignment of LRMs via Counter-Aligned Few-Shot Conversation Exposurecited by 0
  11. 2026Towards Mitigating Deceptive Safety Alignment in Large Reasoning Modelscited by 0
  12. 2026CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025UnsafeChain: Enhancing Reasoning Model Safety via Hard Cases
  2. 2025RAJ-PGA: Reasoning-Activated Jailbreak and Principle-Guided Alignment Framework for Large Reasoning Models
  3. 2025Towards Safe Reasoning in Large Reasoning Models via Corrective Intervention
  4. 2025SafeRBench: Dissecting the Reasoning Safety of Large Language Models
  5. 2025STAIR: Improving Safety Alignment with Introspective Reasoning
  6. 2026Chain of Risk: Safety Failures in Large Reasoning Models and Mitigation via Adaptive Multi-Principle Steering
  7. 2026SFCoT: Safer Chain-of-Thought via Active Safety Evaluation and Calibration
  8. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking