mlsanityData to 7 Oct 2026

SaFeR-ToolKit: Structured Reasoning via Virtual Tool Calling for Multimodal Safety

arXiv 2603.02635 ↗First posted 3 Mar 2026cs.LGcited by 0

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 3 more
Abstract (arXiv)

Vision-language models remain susceptible to multimodal jailbreaks and over-refusal because safety hinges on both visual evidence and user intent, while many alignment pipelines supervise only the final response. To address this, we present SaFeR-ToolKit, which formalizes safety decision-making as a checkable protocol. Concretely, a planner specifies a persona, a Perception → Reasoning → Decision tool set, and a constrained transition graph, while a responder outputs a typed key-value tool trace before the final answer. To make the protocol reliably followed in practice, we train a single policy with a three-stage curriculum (SFT → DPO → GRPO), where GRPO directly supervises tool usage beyond answer-level feedback. Our contributions are two-fold: I. Dataset. The first tool-based safety reasoning dataset, comprising 31,654 examples (SFT 6k, DPO 18.6k, GRPO 6k) plus 1k held-out evaluation. II. Experiments. On Qwen2.5-VL, SaFeR-ToolKit significantly improves Safety/Helpfulness/Reasoning Rigor on 3B (29.39/45.04/4.98 → 84.40/71.13/78.87) and 7B (53.21/52.92/19.26 → 86.34/80.79/85.34), while preserving general capabilities (3B: 58.67 → 59.21; 7B: 66.39 → 66.81). Codes are available at https://github.com/Duebassx/SaFeRToolKit.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 52 of its 77 references that are in the map. First posted on arXiv 3 Mar 2026.

Builds on

52 of 77 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Think-Reflect-Revise: A Policy-Guided Reflective Framework for Safety Alignment in Large Vision Language Models8 Dec
  2. 2025SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimization17 Nov
  3. 2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Models8 Oct
  4. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoning16 May
  5. 2025Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Model10 May
  6. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment22 Apr
  7. 2024OpenAI o1 System Card21 Dec

Other references · 45

  1. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  5. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  6. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  7. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  8. 2025OpenAI GPT-5 System Cardcited by 910
  9. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 380
  10. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 376
  11. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 352
  12. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 208
33 more
  1. 2024ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Modelscited by 204
  2. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  3. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 160
  4. 2024G-Designer: Architecting Multi-agent Communication Topologies via Graph Neural Networkscited by 110
  5. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  6. 2025MM-RLHF: The Next Step Forward in Multimodal LLM Alignmentcited by 79
  7. 2024Safety of Multimodal Large Language Models on Images and Textscited by 70
  8. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  9. 2025Unlocking Multimodal Mathematical Reasoning via Process Reward Modelcited by 64
  10. 2024Multimodal Situational Safetycited by 54
  11. 2024LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Modelscited by 49
  12. 2025SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Rewardcited by 45
  13. 2024Mitigating Object Hallucination in Large Vision-Language Models via Image-Grounded Guidancecited by 44
  14. 2025MM-Verify: Enhancing Multimodal Reasoning with Chain-of-Thought Verificationcited by 39
  15. 2024Align Anything: Training All-Modality Models to Follow Instructions with Language Feedbackcited by 38
  16. 2025Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedbackcited by 27
  17. 2025ShieldGemma 2: Robust and Tractable Image Content Moderationcited by 27
  18. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  19. 2025Equilibrate RLHF: Towards Balancing Helpfulness-Safety Trade-off in Large Language Modelscited by 22
  20. 2025STAR-R1: Spatial TrAnsformation Reasoning by Reinforcing Multimodal LLMscited by 21
  21. 2024UniGuard: Towards Universal Safety Guardrails for Jailbreak Attacks on Multimodal Large Language Modelscited by 18
  22. 2024Empirical Analysis of Large Vision-Language Models against Goal Hijacking via Visual Prompt Injectioncited by 17
  23. 2025SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewardscited by 14
  24. 2025Generative RLHF-V: Learning Principles from Multi-modal Human Preferencecited by 13
  25. 2025Adversary-Aware DPO: Enhancing Safety Alignment in Vision Language Models via Adversarial Trainingcited by 12
  26. 2025Aligning Multimodal LLM with Human Preference: A Surveycited by 12
  27. 2025Are Vision-Language Models Safe in the Wild? A Meme-Based Benchmark Studycited by 12
  28. 2024MMRA: A Benchmark for Evaluating Multi-Granularity and Multi-Image Relational Association Capabilities in Large Visual Language Modelscited by 9
  29. 2024YesBut: A High-Quality Annotated Multimodal Dataset for evaluating Satire Comprehension capability of Vision-Language Modelscited by 9
  30. 2025Better Safe Than Sorry? Overreaction Problem of Vision Language Models in Visual Emergency Recognitioncited by 7
  31. 2024Refusing Safe Prompts for Multi-modal Large Language Modelscited by 6
  32. 2025Multimodal Prompt Injection Attacks: Risks and Defenses for Modern LLMscited by 4
  33. 2025SIA: Enhancing Safety via Intent Awareness for Vision-Language Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025MSR-Align: Policy-Grounded Multimodal Alignment for Safety-Aware Reasoning in Vision-Language Models
  2. 2025VLMGuard-R1: Proactive Safety Alignment for VLMs via Reasoning-Driven Prompt Optimization
  3. 2025Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Models
  4. 2025PRISM: Robust VLM Alignment with Principled Reasoning for Integrated Safety in Multimodality
  5. 2025RedDiffuser: Auditing Multimodal Safety Failures in Vision-Language Models via Reinforced Diffusion
  6. 2026MLLMs Fail to Refuse when Using Tools Agentically
  7. 2025SafeCoT: Improving VLM Safety with Minimal Reasoning
  8. 2025When Safe Unimodal Inputs Collide: Optimizing Reasoning Chains for Cross-Modal Safety in Multimodal Large Language Models