# Skywork-VL Reward: An Effective Reward Model for Multimodal Understanding and Reasoning

arXiv: https://arxiv.org/abs/2505.07263 · HTML: https://mlsanity.com/paper/2505.07263/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 16 of its 51 references that are in the map.
- First posted on arXiv 12 May 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 16 of 16)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [RewardBench: Evaluating Reward Models for Language Modeling](https://mlsanity.com/paper/2403.13787/) (2024)
- [HelpSteer2: Open-source dataset for training top-performing reward models](https://mlsanity.com/paper/2406.08673/) (2024)
- [Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMs](https://mlsanity.com/paper/2406.10216/) (2024)
- [VisualPRM: An Effective Process Reward Model for Multimodal Reasoning](https://mlsanity.com/paper/2503.10291/) (2025)
- [InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model](https://mlsanity.com/paper/2501.12368/) (2025)
- [Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought](https://mlsanity.com/paper/2504.05599/) (2025)
- [Quantile Regression for Distributional Reward Models in RLHF](https://mlsanity.com/paper/2409.10164/) (2024)
- [A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Future](https://mlsanity.com/paper/2504.12328/) (2025)

## Built on by (most cited first, 30 of 32)

- [Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergy](https://mlsanity.com/paper/2507.01352/) (2025)
- [Visual Document Understanding and Reasoning: A Multi-Agent Collaboration Framework with Agent-Wise Adaptive Test-Time Scaling](https://mlsanity.com/paper/2508.03404/) (2025)
- [Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearch](https://mlsanity.com/paper/2512.02395/) (2025)
- [Skywork-R1V3 Technical Report](https://mlsanity.com/paper/2507.06167/) (2025)
- [Test-time scaling of diffusions with flow maps](https://mlsanity.com/paper/2511.22688/) (2025)
- [Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Models](https://mlsanity.com/paper/2505.18536/) (2025)
- [MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoning](https://mlsanity.com/paper/2603.02024/) (2026)
- [TIM-PRM: Verifying multimodal reasoning with Tool-Integrated PRM](https://mlsanity.com/paper/2511.22998/) (2025)
- [VideoRewardBench: Comprehensive Evaluation of Multimodal Reward Models for Video Understanding](https://mlsanity.com/paper/2509.00484/) (2025)
- [VQualA 2025 Challenge on Engagement Prediction for Short Videos: Methods and Results](https://mlsanity.com/paper/2509.02969/) (2025)
- [GGBench: A Geometric Generative Reasoning Benchmark for Unified Multimodal Models](https://mlsanity.com/paper/2511.11134/) (2025)
- [How to Guide Your Flow: Few-Step Alignment via Flow Map Reward Guidance](https://mlsanity.com/paper/2604.27147/) (2026)
- [TAR: Temporal Anchor-Constrained Reasoning for Video Temporal Grounding](https://mlsanity.com/paper/2508.07683/) (2025)
- [Calibrated Test-Time Guidance for Bayesian Inference](https://mlsanity.com/paper/2602.22428/) (2026)
- [MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning](https://mlsanity.com/paper/2603.25108/) (2026)
- [Visual Preference Optimization with Rubric Rewards](https://mlsanity.com/paper/2604.13029/) (2026)
- [Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence](https://mlsanity.com/paper/2608.31075/) (2026)
- [MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answering](https://mlsanity.com/paper/2511.12142/) (2025)
- [Do LLMs and VLMs Share Neurons for Inference? Evidence and Mechanisms of Cross-Modal Transfer](https://mlsanity.com/paper/2602.19058/) (2026)
- [Training Data Efficiency in Multimodal Process Reward Models](https://mlsanity.com/paper/2602.04145/) (2026)
- [Video Understanding Reward Modeling: A Robust Benchmark and Performant Reward Models](https://mlsanity.com/paper/2605.07872/) (2026)
- [Beyond Monolithic Rewards: A Hybrid and Multi-Aspect Reward Optimization for MLLM Alignment](https://mlsanity.com/paper/2510.05283/) (2025)
- [Ostrakon-VL: Towards Domain-Expert MLLM for Food-Service and Retail Stores](https://mlsanity.com/paper/2601.21342/) (2026)
- [From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderation](https://mlsanity.com/paper/2602.02536/) (2026)
- [DesignSense: A Human Preference Dataset and Reward Modeling Framework for Graphic Layout Generation](https://mlsanity.com/paper/2602.23438/) (2026)
- [Learning What Matters: Dynamic Dimension Selection and Aggregation for Interpretable Vision-Language Reward Modeling](https://mlsanity.com/paper/2604.05445/) (2026)
- [You Only Judge Once: Multi-response Reward Modeling in a Single Forward Pass](https://mlsanity.com/paper/2604.10966/) (2026)
- [DT2IT-MRM: Debiased Preference Construction and Iterative Training for Multimodal Reward Modeling](https://mlsanity.com/paper/2604.19544/) (2026)
- [V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning](https://mlsanity.com/paper/2608.25580/) (2026)
- [SpatialSkill: Self-Evolving Skills for Cross-View Spatial Reasoning](https://mlsanity.com/paper/2609.34124/) (2026)

## Abstract (arXiv)

We propose Skywork-VL Reward, a multimodal reward model that provides reward signals for both multimodal understanding and reasoning tasks. Our technical approach comprises two key components: First, we construct a large-scale multimodal preference dataset that covers a wide range of tasks and scenarios, with responses collected from both standard vision-language models (VLMs) and advanced VLM reasoners. Second, we design a reward model architecture based on Qwen2.5-VL-7B-Instruct, integrating a reward head and applying multi-stage fine-tuning using pairwise ranking loss on pairwise preference data. Experimental evaluations show that Skywork-VL Reward achieves state-of-the-art results on multimodal VL-RewardBench and exhibits competitive performance on the text-only RewardBench benchmark. Furthermore, preference data constructed based on our Skywork-VL Reward proves highly effective for training Mixed Preference Optimization (MPO), leading to significant improvements in multimodal reasoning capabilities. Our results underscore Skywork-VL Reward as a significant advancement toward general-purpose, reliable reward models for multimodal alignment. Our model has been publicly released to promote transparency and reproducibility.