mlsanityData to 14 Oct 2026

STAIR: Improving Safety Alignment with Introspective Reasoning

arXiv 2502.02384 ↗First posted 4 Feb 2025cs.CLcited by 77

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 2 more
Abstract (arXiv)

Ensuring the safety and harmlessness of Large Language Models (LLMs) has become equally critical as their performance in applications. However, existing safety alignment methods typically suffer from safety-performance trade-offs and the susceptibility to jailbreak attacks, primarily due to their reliance on direct refusals for malicious queries. In this paper, we propose STAIR, a novel framework that integrates SafeTy Alignment with Itrospective Reasoning. We enable LLMs to identify safety risks through step-by-step analysis by self-improving chain-of-thought (CoT) reasoning with safety awareness. STAIR first equips the model with a structured reasoning capability and then advances safety alignment via iterative preference optimization on step-level reasoning data generated using our newly proposed Safety-Informed Monte Carlo Tree Search (SI-MCTS). We further train a process reward model on this data to guide test-time searches for improved responses. Extensive experiments show that STAIR effectively mitigates harmful outputs while better preserving helpfulness, compared to instinctive alignment strategies. With test-time scaling, STAIR achieves a safety performance comparable to Claude-3.5 against popular jailbreak attacks. Relevant resources in this work are available at https://github.com/thu-ml/STAIR.

Key facts. Cited by 77 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 68 of its 86 references that are in the map. First posted on arXiv 4 Feb 2025.

Builds on

68 of 86 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Deliberative Alignment: Reasoning Enables Safer Language Models20 Dec
  2. 2024Rethinking Machine Unlearning for Large Language Models13 Feb
  3. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan

Other references · 65

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  10. 2023Let's Verify Step by Stepcited by 3,789
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  12. 2023Qwen Technical Reportcited by 3,344
38 more
  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  3. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  4. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  6. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  7. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  8. 2024Self-Rewarding Language Modelscited by 681
  9. 2022Large Language Models Can Self-Improvecited by 664
  10. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  11. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  12. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  13. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  14. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 500
  15. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
  16. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 472
  17. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
  18. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 406
  19. 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 396
  20. 2024A StrongREJECT for Empty Jailbreakscited by 385
  21. 2024Measuring short-form factuality in large language modelscited by 373
  22. 2023What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuningcited by 364
  23. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
  24. 2024Improving Alignment and Robustness with Circuit Breakerscited by 322
  25. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 295
  26. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 232
  27. 2024Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modificationscited by 231
  28. 2023Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theorycited by 230
  29. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 230
  30. 2023Using an LLM to Help With Code Understandingcited by 228
  31. 2024Foundational Challenges in Assuring Alignment and Safety of Large Language Modelscited by 199
  32. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 196
  33. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
  34. 2023Mitigating the Alignment Tax of RLHFcited by 183
  35. 2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 183
  36. 2024AlphaMath Almost Zero: Process Supervision without Processcited by 162
  37. 2024Accessing GPT-4 level Mathematical Olympiad Solutions via Monte Carlo Tree Self-refine with LLaMa-3 8Bcited by 158
  38. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 130

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 42; 2026: 35.

77 papers.

Sort:

By area:

  1. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  2. 2025Safety in Large Reasoning Models: A Surveycited by 64
  3. 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capabilitycited by 59
  4. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safetycited by 39
  5. 2025Mitigating Overthinking in Large Reasoning Models via Manifold Steeringcited by 33
  6. 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agentscited by 29
  7. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  8. 2025Reasoning as an Adaptive Defense for Safetycited by 23
  9. 2025Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignmentcited by 20
  10. 2025DeceptionBench: A Comprehensive Benchmark for AI Deception Behaviors in Real-world Scenarioscited by 19
  11. 2025Breaking the Ceiling: Exploring the Potential of Jailbreak Attacks through Expanding Strategy Spacecited by 17
  12. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
38 more
  1. 2025Lifelong Safety Alignment for Language Modelscited by 15
  2. 2025TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewardscited by 14
  3. 2026CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agentscited by 14
  4. 2025Large Reasoning Models Learn Better Alignment from Flawed Thinkingcited by 13
  5. 2025SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimizationcited by 12
  6. 2025Strata-Sword: A Hierarchical Safety Evaluation towards LLMs based on Reasoning Complexity of Jailbreak Instructionscited by 11
  7. 2025Towards Safe Reasoning in Large Reasoning Models via Corrective Interventioncited by 11
  8. 2025A2D: Any-Order, Any-Step Safety Alignment for Diffusion Language Modelscited by 9
  9. 2025ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoningcited by 8
  10. 2025SARSteer: Safeguarding Large Audio-Language Models via Safe-Ablated Refusal Steeringcited by 8
  11. 2025UnsafeChain: Enhancing Reasoning Model Safety via Hard Casescited by 7
  12. 2025Reasoned Safety Alignment: Ensuring Jailbreak Defense via Answer-Then-Checkcited by 6
  13. 2026Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projectioncited by 6
  14. 2025Strong but Brittle: Simple Attacks Subvert Reasoning-based Safety Guardrailscited by 5
  15. 2025SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoningcited by 4
  16. 2025Unveiling Trust in Multimodal Large Language Models: Evaluation, Analysis, and Mitigationcited by 4
  17. 2025Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberationcited by 4
  18. 2026Defending Large Language Models Against Jailbreak Attacks via In-Decoding Safety-Awareness Probingcited by 4
  19. 2026Overview of PAN 2026: Voight-Kampff Generative AI Detection, Text Watermarking, Multi-Author Writing Style Analysis, Generative Plagiarism Detection, and Reasoning Trajectory Detectioncited by 4
  20. 2025AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applicationscited by 3
  21. 2026STAR-S: Improving Safety Alignment through Self-Taught Reasoning on Safety Rulescited by 3
  22. 2026Mitigating Safety Tax via Distribution-Grounded Refinement in Large Reasoning Modelscited by 3
  23. 2026CAP: Controllable Alignment Prompting for Unlearning in LLMscited by 3
  24. 2025Does More Inference-Time Compute Really Help Robustness?cited by 2
  25. 2025PRISM: Robust VLM Alignment with Principled Reasoning for Integrated Safety in Multimodalitycited by 2
  26. 2025EASE: Practical and Efficient Safety Alignment for Small Language Modelscited by 2
  27. 2025Beyond SFT: Reinforcement Learning for Safer Large Reasoning Models with Better Reasoning Abilitycited by 2
  28. 2026When the Prompt Becomes Visual: Vision-Centric Jailbreak Attacks for Large Image Editing Modelscited by 2
  29. 2026Alignment-Weighted DPO: A principled reasoning approach to improve safety alignmentcited by 2
  30. 2026Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representationscited by 2
  31. 2025Beyond Surface-Level Detection: Towards Cognitive-Driven Defense Against Jailbreak Attacks via Meta-Operations Reasoningcited by 1
  32. 2025NDM: A Noise-driven Detection and Mitigation Framework against Implicit Sexual Intentions in Text-to-Image Generationcited by 1
  33. 2025From Perception to Reasoning: Deep Thinking Empowers Multimodal Large Language Modelscited by 1
  34. 2025Think-Reflect-Revise: A Policy-Guided Reflective Framework for Safety Alignment in Large Vision Language Modelscited by 1
  35. 2026RAPO: Risk-Aware Preference Optimization for Generalizable Safe Reasoningcited by 1
  36. 2026Mitigating Mismatch within Reference-based Preference Optimizationcited by 1
  37. 2026Improving Safety Alignment via Balanced Direct Preference Optimizationcited by 1
  38. 2026Disentangling Intent from Role: Adversarial Self-Play for Persona-Invariant Safety Alignmentcited by 1

50 most cited of 77 All 77 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025SaRO: Enhancing LLM Safety through Reasoning-based Alignment
  2. 2025Decoding One Safety Trigger Token for Balancing Safety and Usability in Large Language Models
  3. 2026MOSAIC: Composable Safety Alignment with Modular Control Tokens
  4. 2024Superficial Safety Alignment Hypothesis
  5. 2025Why Safeguarded Ships Run Aground? Aligned Large Language Models' Safety Mechanisms Tend to Be Anchored in The Template Region
  6. 2025SABER: Uncovering Vulnerabilities in Safety Alignment via Cross-Layer Residual Connection
  7. 2025SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment
  8. 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking