# SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment

arXiv: https://arxiv.org/abs/2505.14667 · HTML: https://mlsanity.com/paper/2505.14667/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 58 of its 60 references that are in the map.
- First posted on arXiv 20 May 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 30 of 58)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Editing Models with Task Arithmetic](https://mlsanity.com/paper/2212.04089/) (2022)
- [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://mlsanity.com/paper/2307.04657/) (2023)
- [Tree of Attacks: Jailbreaking Black-Box LLMs Automatically](https://mlsanity.com/paper/2312.02119/) (2023)
- [GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts](https://mlsanity.com/paper/2309.10253/) (2023)
- [XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models](https://mlsanity.com/paper/2308.01263/) (2023)
- [Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning](https://mlsanity.com/paper/2404.05868/) (2024)
- ["Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models](https://mlsanity.com/paper/2308.03825/) (2023)
- [Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks](https://mlsanity.com/paper/2404.02151/) (2024)
- [Safety Alignment Should Be Made More Than Just a Few Tokens Deep](https://mlsanity.com/paper/2406.05946/) (2024)
- [MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering](https://mlsanity.com/paper/2410.07095/) (2024)
- [A StrongREJECT for Empty Jailbreaks](https://mlsanity.com/paper/2402.10260/) (2024)

## Built on by (most cited first, 24 of 24)

- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [Large Reasoning Models Learn Better Alignment from Flawed Thinking](https://mlsanity.com/paper/2510.00938/) (2025)
- [Distributional AGI Safety](https://mlsanity.com/paper/2512.16856/) (2025)
- [R-TOFU: Unlearning in Large Reasoning Models](https://mlsanity.com/paper/2505.15214/) (2025)
- [A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Models](https://mlsanity.com/paper/2509.23286/) (2025)
- [What Matters For Safety Alignment?](https://mlsanity.com/paper/2601.03868/) (2026)
- [Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden States](https://mlsanity.com/paper/2608.02657/) (2026)
- [When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models](https://mlsanity.com/paper/2510.21285/) (2025)
- [ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Moments](https://mlsanity.com/paper/2508.04204/) (2025)
- [Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberation](https://mlsanity.com/paper/2509.14760/) (2025)
- [Safety Recovery in Reasoning Models Is Only a Few Early Steering Steps Away](https://mlsanity.com/paper/2602.11096/) (2026)
- [Do Thinking Tokens Help with Safety?](https://mlsanity.com/paper/2606.25013/) (2026)
- [Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representations](https://mlsanity.com/paper/2603.17305/) (2026)
- [Towards Safer Large Reasoning Models by Promoting Safety Decision-Making before Chain-of-Thought Generation](https://mlsanity.com/paper/2603.17368/) (2026)
- [Tool Specifications Matter: Uncovering and Mitigating Safety Risks in AI Agents](https://mlsanity.com/paper/2607.29254/) (2026)
- [Concept Concentration for Faithful Representation Intervention](https://mlsanity.com/paper/2505.18672/) (2025)
- [Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models](https://mlsanity.com/paper/2605.24550/) (2026)
- [Dual-Adversarial Safety Alignment: Cultivating Intrinsic Threat Comprehension in LRMs](https://mlsanity.com/paper/2608.09542/) (2026)
- [INTENT-AS-A-TOOL Makes it Easy to Track Agentic Misalignment](https://mlsanity.com/paper/2608.27348/) (2026)
- [Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal](https://mlsanity.com/paper/2609.04482/) (2026)
- [First Token Matters: Understanding Safety Collapse in Large Reasoning Models](https://mlsanity.com/paper/2609.18471/) (2026)
- [Alignment of LRMs via Counter-Aligned Few-Shot Conversation Exposure](https://mlsanity.com/paper/2609.27763/) (2026)
- [Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models](https://mlsanity.com/paper/2609.36254/) (2026)
- [CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completion](https://mlsanity.com/paper/2609.39902/) (2026)

## Abstract (arXiv)

Large Reasoning Models (LRMs) have become powerful tools for complex problem solving, but their structured reasoning pathways can lead to unsafe outputs when exposed to harmful prompts. Existing safety alignment methods reduce harmful outputs but can degrade reasoning depth, leading to significant trade-offs in complex, multi-step tasks, and remain vulnerable to sophisticated jailbreak attacks. To address this, we introduce SAFEPATH, a lightweight alignment method that fine-tunes LRMs to emit a short, 8-token Safety Primer at the start of their reasoning, in response to harmful prompts, while leaving the rest of the reasoning process unsupervised. Empirical results across multiple benchmarks indicate that SAFEPATH effectively reduces harmful outputs while maintaining reasoning performance. Specifically, SAFEPATH reduces harmful responses by up to 90.0% and blocks 83.3% of jailbreak attempts in the DeepSeek-R1-Distill-Llama-8B model, while requiring 295.9x less compute than Direct Refusal and 314.1x less than SafeChain. We further introduce a zero-shot variant that requires no fine-tuning. In addition, we provide a comprehensive analysis of how existing methods in LLMs generalize, or fail, when applied to reasoning-centric models, revealing critical gaps and new directions for safer AI.