From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderation
On the map: LLM safety and security → Vision-language and text-to-image safety
Abstract (arXiv)
Safety moderation is pivotal for identifying harmful content. Despite the success of textual safety moderation, its multimodal counterparts remain hindered by a dual sparsity of data and supervision. Conventional reliance on binary labels lead to shortcut learning, which obscures the intrinsic classification boundaries necessary for effective multimodal discrimination. Hence, we propose a novel learning paradigm (UniMod) that transitions from sparse decision-making to dense reasoning traces. By constructing structured trajectories encompassing evidence grounding, modality assessment, risk mapping, policy decision, and response generation, we reformulate monolithic decision tasks into a multi-dimensional boundary learning process. This approach forces the model to ground its decision in explicit safety semantics, preventing the model from converging on superficial shortcuts. To facilitate this paradigm, we develop a multi-head scalar reward model (UniRM). UniRM provides multi-dimensional supervision by assigning attribute-level scores to the response generation stage. Furthermore, we introduce specialized optimization strategies to decouple task-specific parameters and rebalance training dynamics, effectively resolving interference between diverse objectives in multi-task learning. Empirical results show UniMod achieves competitive textual moderation performance and sets a new multimodal benchmark using less than 40% of the training data used by leading baselines. Ablations further validate our multi-attribute trajectory reasoning, offering an effective and efficient framework for multimodal moderation. Supplementary materials are available at hrefhttps://trustworthylab.github.io/UniMod/project website.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 56 of its 66 references that are in the map. First posted on arXiv 28 Jan 2026.
Builds on
56 of 66 references.
Closest ancestors · 7
Same direction, newest first
- 2025JailBound: Jailbreaking Internal Safety Boundaries of Vision-Language Models26 May
- 2024Zer0-Jack: A Memory-efficient Gradient-based Jailbreaking Method for Black-box Multi-modal Large Language Models12 Nov
- 2024Safe Inputs but Unsafe Output: Benchmarking Cross-modality Safety Alignment of Large Vision-Language Model21 Jun
- 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model17 Jun
- 2024White-box Multimodal Jailbreaks Against Large Vision-Language Models28 May
- 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models3 Feb
- 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
Other references · 49
- 2023GPT-4 Technical Reportcited by 18,132
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
37 more
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
- 2023A Survey on Multimodal Large Language Modelscited by 898
- 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
- 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
- 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
- 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
- 2024Deliberative Alignment: Reasoning Enables Safer Language Modelscited by 292
- 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
- 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 255
- 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 224
- 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 224
- 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
- 2023Making LLaMA SEE and Draw with SEED Tokenizercited by 184
- 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
- 2025Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learningcited by 159
- 2025RewardBench 2: Advancing Reward Model Evaluationcited by 136
- 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrailscited by 122
- 2024Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversationscited by 118
- 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attackscited by 89
- 2023Rethinking Model Ensemble in Transfer-based Adversarial Attackscited by 88
- 2024Safety of Multimodal Large Language Models on Images and Textscited by 70
- 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vectorcited by 63
- 2024Red Teaming Visual Language Modelscited by 62
- 2025Hume: Introducing System-2 Thinking in Visual-Language-Action Modelcited by 56
- 2024An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Modelscited by 40
- 2024MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Modelscited by 34
- 2020Learning Black-Box Attackers with Transferable Priors and Query Feedbackcited by 32
- 2025Skywork-VL Reward: An Effective Reward Model for Multimodal Understanding and Reasoningcited by 30
- 2025Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedbackcited by 27
- 2025Oyster-I: Beyond Refusal -- Constructive Safety Alignment for Responsible Language Modelscited by 19
- 2026OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMscited by 18
- 2024Chain of History: Learning and Forecasting with LLMs for Temporal Knowledge Graph Completioncited by 10
- 2025USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Modelscited by 10
- 2025Bradley-Terry and Multi-Objective Reward Modeling Are Complementarycited by 10
- 2026Reward Modeling from Natural Language Human Feedbackcited by 7
- 2024Smaller Large Language Models Can Do Moral Self-Correctioncited by 5
- 2025ProGuard: Towards Proactive Multimodal Safeguardcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MuPHI: Learning Implicit Multimodal Harm Reasoning via Semantically Grounded Reward Optimization
- 2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Models
- 2025Towards Trustworthy Multimodal Moderation via Policy-Aligned Reasoning and Hierarchical Labeling
- 2025When Safe Unimodal Inputs Collide: Optimizing Reasoning Chains for Cross-Modal Safety in Multimodal Large Language Models
- 2025SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimization
- 2026RuleSafe-VL: Evaluating Rule-Conditioned Decision Reasoning in Vision-Language Content Moderation
- 2026ThinkingGuard: Decoding Implicit Hazards via Step-by-Step Risk Attribution in Multimodal Large Language Models
- 2026Nemotron 3.5 Content Safety Moderator: A Compact Multimodal, Multilingual, and Reasoning Enabled Content Safety Moderator