mlsanityData to 14 Oct 2026

SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning

arXiv 2505.16186 ↗First posted 22 May 2025cs.AI cs.CL cs.CRcited by 20

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Large Reasoning Models (LRMs) introduce a new generation paradigm of explicitly reasoning before answering, leading to remarkable improvements in complex tasks. However, they pose great safety risks against harmful queries and adversarial attacks. While recent mainstream safety efforts on LRMs, supervised fine-tuning (SFT), improve safety performance, we find that SFT-aligned models struggle to generalize to unseen jailbreak prompts. After thorough investigation of LRMs' generation, we identify a safety aha moment that can activate safety reasoning and lead to a safe response. This aha moment typically appears in the `key sentence', which follows models' query understanding process and can indicate whether the model will proceed safely. Based on these insights, we propose SafeKey, including two complementary objectives to better activate the safety aha moment in the key sentence: (1) a Dual-Path Safety Head to enhance the safety signal in the model's internal representations before the key sentence, and (2) a Query-Mask Modeling objective to improve the models' attention on its query understanding, which has important safety hints. Experiments across multiple safety benchmarks demonstrate that our methods significantly improve safety generalization to a wide range of jailbreak attacks and out-of-distribution harmful prompts, lowering the average harmfulness rate by 9.6%, while maintaining general abilities. Our analysis reveals how SafeKey enhances safety by reshaping internal attention and improving the quality of hidden representations.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 29 of its 38 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

29 of 38 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capability14 Apr
  2. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Data2 Apr
  3. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinking18 Feb
  4. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R118 Feb
  5. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities17 Feb
  6. 2024Deliberative Alignment: Reasoning Enables Safer Language Models20 Dec

Other references · 23

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  6. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  7. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  8. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  9. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  10. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  11. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  12. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
11 more
  1. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
  2. 2024A StrongREJECT for Empty Jailbreakscited by 385
  3. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 199
  4. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 125
  5. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 107
  6. 2024GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysiscited by 96
  7. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 86
  8. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 69
  9. 2025Improving LLM Safety Alignment with Dual-Objective Optimizationcited by 43
  10. 2024Backtracking Improves Generation Safetycited by 32
  11. 2024From Theft to Bomb-Making: The Ripple Effect of Unlearning in Defending Against Jailbreak Attackscited by 29

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 14; 2026: 6.

20 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  2. 2025Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Modelscited by 17
  3. 2025Large Reasoning Models Learn Better Alignment from Flawed Thinkingcited by 13
  4. 2025Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Trainingcited by 12
  5. 2025Towards Safe Reasoning in Large Reasoning Models via Corrective Interventioncited by 11
  6. 2026Reducing the Safety Tax in LLM Safety Alignment with On-Policy Self-Distillationcited by 11
  7. 2025Asynchronous Reasoning: Training-Free Interactive Thinking LLMscited by 7
  8. 2025R1-ACT: Efficient Reasoning Model Safety Alignment by Activating Safety Knowledgecited by 6
  9. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Modelscited by 6
  10. 2025Strong but Brittle: Simple Attacks Subvert Reasoning-based Safety Guardrailscited by 5
  11. 2026Endogenous Resistance to Activation Steering in Language Modelscited by 5
  12. 2025ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Momentscited by 4
8 more
  1. 2025AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Modelscited by 4
  2. 2026Do Thinking Tokens Help with Safety?cited by 3
  3. 2025Red Teaming Large Reasoning Modelscited by 2
  4. 2025Learning to Extract Context for Context-Aware LLM Inferencecited by 2
  5. 2026Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representationscited by 2
  6. 2026Self-ReSET: Learning to Self-Recover from Unsafe Reasoning Trajectoriescited by 1
  7. 2025UpSafe^circC: Upcycling for Controllable Safety in Large Language Modelscited by 0
  8. 2026Addressing Over-Refusal in LLMs with Competing Rewardscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models
  2. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Models
  3. 2025How Should We Enhance the Safety of Large Reasoning Models: An Empirical Study
  4. 2026Beyond Content Safety: Real-Time Monitoring for Reasoning Vulnerabilities in Large Language Models
  5. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety
  6. 2025SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models
  7. 2025Safety Alignment Can Be Not Superficial With Explicit Safety Signals
  8. 2025SafeRBench: Dissecting the Reasoning Safety of Large Language Models