# RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards

arXiv: https://arxiv.org/abs/2506.07736 · HTML: https://mlsanity.com/paper/2506.07736/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 21 of its 77 references that are in the map.
- First posted on arXiv 9 Jun 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM alignment / RLHF / preference optimisation / [Safety alignment and jailbreak attacks](https://mlsanity.com/alignment/safety-alignment-and-jailbreak-attacks/)
- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 21 of 21)

- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025)
- [RRHF: Rank Responses to Align Language Models with Human Feedback without tears](https://mlsanity.com/paper/2304.05302/) (2023)
- [Recursively Summarizing Books with Human Feedback](https://mlsanity.com/paper/2109.10862/) (2021)
- [PKU-SafeRLHF: Towards Multi-Level Safety Alignment for LLMs with Human Preference](https://mlsanity.com/paper/2406.15513/) (2024)
- [OR-Bench: An Over-Refusal Benchmark for Large Language Models](https://mlsanity.com/paper/2405.20947/) (2024)
- [AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models](https://mlsanity.com/paper/2410.02355/) (2024)
- [Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't](https://mlsanity.com/paper/2503.16219/) (2025)
- [Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreaks](https://mlsanity.com/paper/2411.16721/) (2024)
- [Thinking LLMs: General Instruction Following with Thought Generation](https://mlsanity.com/paper/2410.10630/) (2024)
- [R2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning](https://mlsanity.com/paper/2407.05557/) (2024)
- [AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender](https://mlsanity.com/paper/2504.09466/) (2025)
- [SeaLLMs 3: Open Foundation and Chat Multilingual Large Language Models for Southeast Asian Languages](https://mlsanity.com/paper/2407.19672/) (2024)
- [SafeMLRM: Demystifying Safety in Multi-modal Large Reasoning Models](https://mlsanity.com/paper/2504.08813/) (2025)
- [Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game](https://mlsanity.com/paper/2311.08045/) (2023)
- [Towards Comprehensive Post Safety Alignment of Large Language Models via Safety Patching](https://mlsanity.com/paper/2405.13820/) (2024)
- [AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement](https://mlsanity.com/paper/2502.16776/) (2025)
- [Multilingual Performance Biases of Large Language Models in Education](https://mlsanity.com/paper/2504.17720/) (2025)
- [Assessing the Capability of Large Language Models for Domain-Specific Ontology Generation](https://mlsanity.com/paper/2504.17402/) (2025)
- [AI-Enhanced Business Process Automation: A Case Study in the Insurance Domain Using Object-Centric Process Mining](https://mlsanity.com/paper/2504.17295/) (2025)

## Built on by (most cited first, 14 of 14)

- [The Landscape of Agentic Reinforcement Learning for LLMs: A Survey](https://mlsanity.com/paper/2509.02547/) (2025)
- [A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Models](https://mlsanity.com/paper/2509.03871/) (2025)
- [Risky-Bench: Probing Agentic Safety Risks under Real-World Deployment](https://mlsanity.com/paper/2602.03100/) (2026)
- [What Matters For Safety Alignment?](https://mlsanity.com/paper/2601.03868/) (2026)
- [Mi:dm 2.0 Korea-centric Bilingual Language Models](https://mlsanity.com/paper/2601.09066/) (2026)
- [IntentionReasoner: Facilitating Adaptive LLM Safeguards through Intent Reasoning and Selective Query Refinement](https://mlsanity.com/paper/2508.20151/) (2025)
- [ProGuard: Towards Proactive Multimodal Safeguard](https://mlsanity.com/paper/2512.23573/) (2025)
- [DecepChain: Inducing Deceptive Reasoning in Large Language Models](https://mlsanity.com/paper/2510.00319/) (2025)
- [WebAgentGuard: A Reasoning-Driven Guard Model for Detecting Prompt Injection Attacks in Web Agents](https://mlsanity.com/paper/2604.12284/) (2026)
- [Internalizing Safety Understanding in Large Reasoning Models via Verification](https://mlsanity.com/paper/2605.08930/) (2026)
- [Safety Compliance: Rethinking LLM Safety Reasoning through the Lens of Compliance](https://mlsanity.com/paper/2509.22250/) (2025)
- [Alignment-Weighted DPO: A principled reasoning approach to improve safety alignment](https://mlsanity.com/paper/2602.21346/) (2026)
- [When AI Fails, What Works? A Data-Driven Taxonomy of Real-World AI Risk Mitigation Strategies](https://mlsanity.com/paper/2603.04259/) (2026)
- [Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoning](https://mlsanity.com/paper/2508.03054/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) continue to exhibit vulnerabilities despite deliberate safety alignment efforts, posing significant risks to users and society. To safeguard against the risk of policy-violating content, system-level moderation via external guard models-designed to monitor LLM inputs and outputs and block potentially harmful content-has emerged as a prevalent mitigation strategy. Existing approaches of training guard models rely heavily on extensive human curated datasets and struggle with out-of-distribution threats, such as emerging harmful categories or jailbreak attacks. To address these limitations, we propose RSafe, an adaptive reasoning-based safeguard that conducts guided safety reasoning to provide robust protection within the scope of specified safety policies. RSafe operates in two stages: 1) guided reasoning, where it analyzes safety risks of input content through policy-guided step-by-step reasoning, and 2) reinforced alignment, where rule-based RL optimizes its reasoning paths to align with accurate safety prediction. This two-stage training paradigm enables RSafe to internalize safety principles to generalize safety protection capability over unseen or adversarial safety violation scenarios. During inference, RSafe accepts user-specified safety policies to provide enhanced safeguards tailored to specific safety requirements.