mlsanityData to 7 Oct 2026

Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning

arXiv 2504.16656 ↗First posted 23 Apr 2025cs.CVcited by 48

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

We present Skywork R1V2, a next-generation multimodal reasoning model and a major leap forward from its predecessor, Skywork R1V. At its core, R1V2 introduces a hybrid reinforcement learning paradigm that jointly leverages the Mixed Preference Optimization (MPO) and the Group Relative Policy Optimization (GRPO), which harmonizes reward-model guidance with rule-based strategies, thereby addressing the long-standing challenge of balancing sophisticated reasoning capabilities with broad generalization. To further enhance training efficiency, we propose the Selective Sample Buffer (SSB) mechanism, which effectively addresses the vanishing advantages dilemma inherent in GRPO by prioritizing high-value samples throughout the optimization process. Notably, we observe that excessive reinforcement signals can induce visual hallucinations--a phenomenon we systematically monitor and mitigate through calibrated reward thresholds throughout the training process. Empirical results affirm the exceptional capability of R1V2, with benchmark-leading performances such as 62.6 on OlympiadBench, 78.9 on AIME2024, 63.6 on LiveCodeBench, and 73.6 on MMMU. These results underscore R1V2's superiority over existing open-source models and demonstrate significant progress in closing the performance gap with premier proprietary systems, including Gemini 2.5 and OpenAI-o4-mini. The Skywork R1V2 model weights have been publicly released to promote openness and reproducibility https://huggingface.co/Skywork/Skywork-R1V2-38B.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 51 references that are in the map. First posted on arXiv 23 Apr 2025.

Builds on

38 of 51 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models10 Apr
  2. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
  3. 2025Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought8 Apr
  4. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization15 Nov
  5. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  6. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  7. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action20 Mar
  8. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar

Other references · 30

  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  5. 2025Qwen2.5-VL Technical Reportcited by 5,418
  6. 2017Deep reinforcement learning from human preferencescited by 3,846
  7. 2023Qwen Technical Reportcited by 3,273
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  9. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  10. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  11. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  12. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
18 more
  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  2. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  3. 2019Towards VQA Models That Can Readcited by 1,696
  4. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
  5. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  6. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  7. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  8. 2016A Diagram Is Worth A Dozen Imagescited by 793
  9. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  10. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  11. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  12. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 283
  13. 2023Mitigating the Alignment Tax of RLHFcited by 181
  14. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 180
  15. 2024Direct Preference Optimization of Video Large Multimodal Models from Language Model Rewardcited by 145
  16. 2024Binary Classifier Optimization for Large Language Model Alignmentcited by 44
  17. 2024Mixed Preference Optimization: Reinforcement Learning with Data Selection and Better Reference Modelcited by 25
  18. 2024Improving Reinforcement Learning from Human Feedback Using Contrastive Rewardscited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 27; 2026: 21.

48 papers.

Sort:

By area:

  1. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  2. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  3. 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 95
  4. 2025R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPOcited by 52
  5. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 43
  6. 2025Spotlight on Token Perception for Multimodal Reinforcement Learningcited by 37
  7. 2025VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoningcited by 26
  8. 2025Skywork-R1V4: Toward Agentic Multimodal Intelligence through Interleaved Thinking with Images and DeepResearchcited by 25
  9. 2025Rex-Thinker: Grounded Object Referring via Chain-of-Thought Reasoningcited by 23
  10. 2025WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learningcited by 21
  11. 2025ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Mapscited by 20
  12. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
36 more
  1. 2025Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Modelscited by 16
  2. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 13
  3. 2025The Synergy Dilemma of Long-CoT SFT and RL: Investigating Post-Training Techniques for Reasoning VLMscited by 12
  4. 2026Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMscited by 9
  5. 2025ProxyThinker: Test-Time Guidance through Small Visual Reasonerscited by 8
  6. 2025Reinforcement Learning for Large Model: A Surveycited by 8
  7. 2025Elicit and Enhance: Advancing Multimodal Reasoning in Medical Scenarioscited by 6
  8. 2026CogFlow: Bridging Perception and Reasoning through Knowledge Internalization for Visual Mathematical Problem Solvingcited by 5
  9. 2025Boosting Reasoning in Large Multimodal Models via Activation Replaycited by 4
  10. 2025AMUSE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understandingcited by 4
  11. 2026Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimizationcited by 4
  12. 2026PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshapingcited by 4
  13. 2025VTPerception-R1: Enhancing Multimodal Reasoning via Explicit Visual and Textual Perceptual Groundingcited by 3
  14. 2025ReVSeg: Incentivizing the Reasoning Chain for Video Segmentation with Reinforcement Learningcited by 3
  15. 2026Distilling the Thought, Watermarking the Answer: A Principle Semantic Guided Watermark for Large Reasoning Modelscited by 3
  16. 2026A Visual Semantic Adaptive Watermark grounded by Prefix-Tuning for Large Vision-Language Modelcited by 3
  17. 2026What does RL improve for Visual Reasoning? A Frankenstein-Style Analysiscited by 3
  18. 2026Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Docited by 3
  19. 2025FinLMM-R1: Enhancing Financial Reasoning in LMM through Scalable Data and Reward Designcited by 2
  20. 2026R^3: Replay, Reflection, and Ranking Rewards for LLM Reinforcement Learningcited by 2
  21. 2026Recursive Think-Answer Process for LLMs and VLMscited by 2
  22. 2025Clarification as Supervision: Reinforcement Learning for Vision-Language Interfacescited by 1
  23. 2025More Than the Final Answer: Improving Visual Extraction and Logical Consistency in Vision-Language Modelscited by 1
  24. 2026CoEditor++: Instruction-based Visual Editing via Cognitive Reasoningcited by 1
  25. 2026Beyond Where to Look: Trajectory-Guided Reinforcement Learning for Multimodal RLVRcited by 1
  26. 2026MMCORE: MultiModal COnnection with Representation Aligned Latent Embeddingscited by 1
  27. 2026Structured Role-Aware Policy Optimization for Multimodal Reasoningcited by 1
  28. 2026Faithful-MR1: Faithful Multimodal Reasoning via Anchoring and Reinforcing Visual Attentioncited by 1
  29. 2025Closing the Gap: Data-Centric Fine-Tuning of Vision Language Models for the Standardized Exam Questionscited by 0
  30. 2025Training Multimodal Large Reasoning Models Needs Better Thoughts: A Three-Stage Framework for Long Chain-of-Thought Synthesis and Selectioncited by 0
  31. 2026Ostrakon-VL: Towards Domain-Expert MLLM for Food-Service and Retail Storescited by 0
  32. 2026Bridging Efficiency and Transparency: Explainable CoT Compression in Multimodal Large Reasoning Modelscited by 0
  33. 2026VeriEvol: Scaling Multimodal Mathematical Reasoning via Verifiable Evol-Instructcited by 0
  34. 2026OracleAnalyser: Analysing Implicit Semantics of Oracle Bone Scripts through MLLMs with Post-trainingcited by 0
  35. 2026Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RLcited by 0
  36. 2026AffectOmni: RL-Verifiable People-Centric Grounded Affective Reasoning for Social and Art-Related Scenescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026OpenVLThinkerV2: A Generalist Multimodal Reasoning Model for Multi-domain Visual Tasks
  2. 2025MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resources
  3. 2025SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning
  4. 2026SVR-R1: Bootstrapping Multi-modal Reasoning with Self-verification in Reinforcement Learning
  5. 2026CFPO: Counterfactual Policy Optimization for Multimodal Reasoning
  6. 2026Rethinking Reinforcement Fine-Tuning in LVLM: Convergence, Reward Decomposition, and Generalization
  7. 2025Skywork-VL Reward: An Effective Reward Model for Multimodal Understanding and Reasoning
  8. 2026All Roads Lead to Rome: Incentivizing Divergent Thinking in Vision-Language Models