mlsanityData to 7 Oct 2026

First Token Matters: Understanding Safety Collapse in Large Reasoning Models

arXiv 2609.18471 ↗First posted 16 Sep 2026cs.AIcited by 0

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 2 more
Abstract (arXiv)

Large Reasoning Models (LRMs) exhibit strong problem-solving abilities, yet their safety alignment often degrades when handling harmful queries. Existing approaches to improving safety largely rely on additional training or preference optimization, while offering limited understanding of the internal mechanisms behind safety failures. In this work, we investigate this failure through a token-level positional analysis of refusal dynamics and identify a localized vulnerability at the onset of reasoning, which we term Onset Refusal Collapse (ORC). We find that the refusal-related signal of LRMs drops sharply at the first generated token under harmful queries, which is associated with unsafe response generation. Motivated by this finding, we propose SafeToken, a lightweight inference-time intervention that injects a learned continuous safety anchor precisely at reasoning onset. Despite updating only a single token embedding, SafeToken effectively mitigates ORC, improves safety on harmful-query benchmarks, and largely preserves reasoning utility. These results suggest that safety failures in LRMs can arise from a transient breakdown at the critical transition from understanding to generation.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 26 of its 46 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

26 of 46 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026Safety Recovery in Reasoning Models Is Only a Few Early Steering Steps Away11 Feb
  2. 2025Chain-of-Thought Hijacking30 Oct
  3. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models24 Oct
  4. 2025Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Training23 Oct
  5. 2025Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?7 Oct
  6. 2025Large Reasoning Models Learn Better Alignment from Flawed Thinking1 Oct
  7. 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment20 May
  8. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Data2 Apr
  9. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking18 Feb
  10. 2024OpenAI o1 System Card21 Dec

Other references · 16

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2023Let's Verify Step by Stepcited by 3,598
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2024Qwen2 Technical Reportcited by 2,076
  6. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  7. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  8. 2024A StrongREJECT for Empty Jailbreakscited by 352
  9. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 206
  10. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 89
  11. 2024Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neuronscited by 47
  12. 2025SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signalscited by 29
4 more
  1. 2025PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Modelscited by 16
  2. 2025Universal Jailbreak Suffixes Are Strong Attention Hijackerscited by 9
  3. 2024Safety Alignment of Large Language Models via Contrasting Safe and Harmful Distributionscited by 8
  4. 2025Read the Scene, Not the Script: Outcome-Aware Safety for LLMscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Chain of Risk: Safety Failures in Large Reasoning Models and Mitigation via Adaptive Multi-Principle Steering
  2. 2025SafeRBench: Dissecting the Reasoning Safety of Large Language Models
  3. 2026Do Thinking Tokens Help with Safety?
  4. 2026DeShortcut-Align: Decoupling Spurious Shortcuts for Robust Safety Alignment in Large Reasoning Models
  5. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety
  6. 2025Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Models
  7. 2025Process or Result? Manipulated Ending Tokens Can Mislead Reasoning LLMs to Ignore the Correct Reasoning Steps
  8. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Models