mlsanityData to 7 Oct 2026

Skywork-VL Reward: An Effective Reward Model for Multimodal Understanding and Reasoning

arXiv 2505.07263 ↗First posted 12 May 2025cs.CVcited by 30

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

We propose Skywork-VL Reward, a multimodal reward model that provides reward signals for both multimodal understanding and reasoning tasks. Our technical approach comprises two key components: First, we construct a large-scale multimodal preference dataset that covers a wide range of tasks and scenarios, with responses collected from both standard vision-language models (VLMs) and advanced VLM reasoners. Second, we design a reward model architecture based on Qwen2.5-VL-7B-Instruct, integrating a reward head and applying multi-stage fine-tuning using pairwise ranking loss on pairwise preference data. Experimental evaluations show that Skywork-VL Reward achieves state-of-the-art results on multimodal VL-RewardBench and exhibits competitive performance on the text-only RewardBench benchmark. Furthermore, preference data constructed based on our Skywork-VL Reward proves highly effective for training Mixed Preference Optimization (MPO), leading to significant improvements in multimodal reasoning capabilities. Our results underscore Skywork-VL Reward as a significant advancement toward general-purpose, reliable reward models for multimodal alignment. Our model has been publicly released to promote transparency and reproducibility.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 51 references that are in the map. First posted on arXiv 12 May 2025.

Builds on

16 of 51 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought8 Apr
  2. 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoning13 Mar
  3. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model21 Jan
  4. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct

Other references · 11

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  6. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  7. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  8. 2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 232
  9. 2024Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMscited by 142
  10. 2024Quantile Regression for Distributional Reward Models in RLHFcited by 59
  11. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 13; 2026: 17.

30 papers.

Sort:

By area:

  1. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  2. 2025Visual Document Understanding and Reasoning: A Multi-Agent Collaboration Framework with Agent-Wise Adaptive Test-Time Scalingcited by 30
  3. 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearchcited by 25
  4. 2025Skywork-R1V3 Technical Reportcited by 16
  5. 2025Test-time scaling of diffusions with flow mapscited by 15
  6. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 13
  7. 2025TIM-PRM: Verifying multimodal reasoning with Tool-Integrated PRMcited by 8
  8. 2026MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoningcited by 8
  9. 2025VQualA 2025 Challenge on Engagement Prediction for Short Videos: Methods and Resultscited by 6
  10. 2025GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Modelscited by 6
  11. 2025TAR: Temporal Anchor-Constrained Reasoning for Video Temporal Groundingcited by 5
  12. 2025VideoRewardBench: Comprehensive Evaluation of Multimodal Reward Models for Video Understandingcited by 5
18 more
  1. 2026MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learningcited by 5
  2. 2026Visual Preference Optimization with Rubric Rewardscited by 5
  3. 2026How to Guide Your Flow: Few-Step Alignment via Flow Map Reward Guidancecited by 5
  4. 2026Calibrated Test-Time Guidance for Bayesian Inferencecited by 4
  5. 2025MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answeringcited by 3
  6. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 2
  7. 2026Training Data Efficiency in Multimodal Process Reward Modelscited by 1
  8. 2026Do LLMs and VLMs Share Neurons for Inference? Evidence and Mechanisms of Cross-Modal Transfercited by 1
  9. 2025Beyond Monolithic Rewards: A Hybrid and Multi-Aspect Reward Optimization for MLLM Alignmentcited by 0
  10. 2026Ostrakon-VL: Towards Domain-Expert MLLM for Food-Service and Retail Storescited by 0
  11. 2026From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderationcited by 0
  12. 2026DesignSense: A Human Preference Dataset and Reward Modeling Framework for Graphic Layout Generationcited by 0
  13. 2026Learning What Matters: Dynamic Dimension Selection and Aggregation for Interpretable Vision-Language Reward Modelingcited by 0
  14. 2026You Only Judge Once: Multi-response Reward Modeling in a Single Forward Passcited by 0
  15. 2026DT2IT-MRM: Debiased Preference Construction and Iterative Training for Multimodal Reward Modelingcited by 0
  16. 2026Video Understanding Reward Modeling: A Robust Benchmark and Performant Reward Modelscited by 0
  17. 2026V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learningcited by 0
  18. 2026SpatialSkill: Self-Evolving Skills for Cross-View Spatial Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Models
  2. 2024Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning
  3. 2025Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning
  4. 2025VLRMBench: A Comprehensive and Challenging Benchmark for Vision-Language Reward Models
  5. 2025Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image
  6. 2026DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification
  7. 2025ViLBench: A Suite for Vision-Language Process Reward Modeling
  8. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward