mlsanityData to 14 Oct 2026

SafeThinker: Reasoning about Risk to Deepen Safety Beyond Shallow Alignment

arXiv 2601.16506 ↗First posted 23 Jan 2026cs.CR cs.AIcited by 1

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 2 more
Abstract (arXiv)

Despite the intrinsic risk-awareness of Large Language Models (LLMs), current defenses often result in shallow safety alignment, rendering models vulnerable to disguised attacks (e.g., prefilling) while degrading utility. To bridge this gap, we propose SafeThinker, an adaptive framework that dynamically allocates defensive resources via a lightweight gateway classifier. Based on the gateway's risk assessment, inputs are routed through three distinct mechanisms: (i) a Standardized Refusal Mechanism for explicit threats to maximize efficiency; (ii) a Safety-Aware Twin Expert (SATE) module to intercept deceptive attacks masquerading as benign queries; and (iii) a Distribution-Guided Think (DDGT) component that adaptively intervenes during uncertain generation. Experiments show that SafeThinker significantly lowers attack success rates across diverse jailbreak strategies without compromising utility, demonstrating that coordinating intrinsic judgment throughout the generation process effectively balances robustness and practicality.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 25 of its 45 references that are in the map. First posted on arXiv 23 Jan 2026.

Builds on

25 of 45 references.

References · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2024Qwen2.5 Technical Reportcited by 4,942
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  8. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  9. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  10. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  11. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  12. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
13 more
  1. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
  2. 2023Detecting Language Model Attacks with Perplexitycited by 431
  3. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  4. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  5. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 306
  6. 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 244
  7. 2024ReFT: Representation Finetuning for Language Modelscited by 197
  8. 2024Weak-to-Strong Jailbreaking on Large Language Modelscited by 107
  9. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 98
  10. 2024Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templatescited by 96
  11. 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysiscited by 64
  12. 2024EasyJailbreak: A Unified Framework for Jailbreaking Large Language Modelscited by 60
  13. 2025JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steeringcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Quantifying Trust: Financial Risk Management for Trustworthy AI Agentscited by 8

Similar, not cited

Close in wording, no citation either way

  1. 2026Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack
  2. 2026Beyond Routine Compliance: Cunning Data Cultivates Safety Vigilance in Large Language Models
  3. 2026Adaptive and Explicit safe: Triggering Latent Safety Awareness in Large Reasoning Models
  4. 2025SafeBehavior: Simulating Human-Like Multistage Reasoning to Mitigate Jailbreak Attacks in Large Language Models
  5. 2025STAIR: Improving Safety Alignment with Introspective Reasoning
  6. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safety
  7. 2025RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards
  8. 2025EASE: Practical and Efficient Safety Alignment for Small Language Models