mlsanityData to 14 Oct 2026

Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models

arXiv 2606.16808 ↗First posted 15 Jun 2026cs.AIcited by 0

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 2 more
Abstract (arXiv)

While Large Reasoning Models (LRMs) excel at complex tasks, they remain highly vulnerable to sophisticated jailbreaks and direct harmful queries. To address this vulnerability, prior works depend heavily on external manual data annotation for safety alignment. However, we observe that LRMs can inherently identify safety risks when being re-presented with original queries alongside their own reasoning trajectories -- a capability we term Latent Safety Awareness. To leverage this safety awareness, we first employ Supervised Fine-Tuning (SFT) to explicitly induce safe tags to trigger safety analysis and guidance following the initial reasoning content for unsafe queries, while preserving standard responses for general queries to ensure adaptive triggering. Subsequently, we apply Direct Preference Optimization (DPO) to further enhance the correctness and stability of the safety analysis and guidance. Notably, responses required for both training stages are entirely generated by models being optimized. With (Safe Trigger) SFT and DPO, experimental results demonstrate significant safety enhancement. For example, the Attack Success Rate (ASR) of DeepSeek-R1-Distill-Llama-8B, on average, drops 24.65% and 36.72% on harmful and jailbreak benchmarks, respectively. Finally, our Safe Trigger method exerts almost no negative impact on general performance or user experience.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 32 of its 48 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

32 of 48 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025LoRA is All You Need for Safety Alignment of Reasoning LLMs22 Jul
  2. 2025ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Models22 May
  3. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Data2 Apr
  4. 2025Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behavior22 Mar
  5. 2025Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable1 Mar
  6. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R118 Feb
  7. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilities17 Feb
  8. 2024OpenAI o1 System Card21 Dec
  9. 2024Deliberative Alignment: Reasoning Enables Safer Language Models20 Dec
  10. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan

Other references · 22

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  6. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  7. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  8. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  9. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  10. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
  11. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 451
  12. 2024PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preferencecited by 261
10 more
  1. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 223
  2. 2019WinoGrande: An Adversarial Winograd Schema Challenge at Scalecited by 204
  3. 2022Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLPcited by 102
  4. 2025Improving LLM Safety Alignment with Dual-Objective Optimizationcited by 43
  5. 2025SaLoRA: Safety-Alignment Preserved Low-Rank Adaptationcited by 42
  6. 2024Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflectioncited by 41
  7. 2025Think Twice, Generate Once: Safeguarding by Progressive Self-Reflectioncited by 11
  8. 2025Module-Aware Parameter-Efficient Machine Unlearning on Transformerscited by 3
  9. 2025Towards Evaluation for Real-World LLM Unlearningcited by 2
  10. 2026Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimizationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning
  2. 2025Towards Safe Reasoning in Large Reasoning Models via Corrective Intervention
  3. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Models
  4. 2025How Should We Enhance the Safety of Large Reasoning Models: An Empirical Study
  5. 2025Self-Guided Adaptive Safety Alignment: Synthesizing and Internalizing Guidelines in Reasoning Models
  6. 2026Reasoning Structure Matters for Safety Alignment of Reasoning Models
  7. 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Ability
  8. 2025When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models