mlsanityData to 7 Oct 2026

VRPRM: Process Reward Modeling via Visual Reasoning

arXiv 2508.03556 ↗First posted 5 Aug 2025cs.LGcited by 11

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Process Reward Model (PRM) is widely used in the post-training of Large Language Model (LLM) because it can perform fine-grained evaluation of the reasoning steps of generated content. However, most PRMs lack long-term reasoning and deep thinking capabilities. On the other hand, although a few works have tried to introduce Chain-of-Thought (CoT) capability into PRMs, the annotation cost of CoT-PRM data is too expensive to play a stable role in various tasks. To address the above challenges, we propose VRPRM, a process reward model via visual reasoning, and design an efficient two-stage training strategy. Experimental results show that using only 3.6K CoT-PRM Supervised Fine-Tuning(SFT) data and 50K non-CoT PRM Reinforcement Learning (RL) training data, VRPRM can surpass the non-thinking PRM with a total data volume of 400K and achieved a relative performance improvement of up to 118% over the base model in the BoN experiment. This result confirms that the proposed combined training strategy can achieve higher quality reasoning capabilities at a lower data annotation cost, thus providing a new paradigm for PRM training with more efficient data utilization.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 19 of its 24 references that are in the map. First posted on arXiv 5 Aug 2025.

Builds on

19 of 24 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models22 May
  2. 2025Reward Reasoning Model20 May
  3. 2025RM-R1: Reward Modeling as Reasoning5 May
  4. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning1 Apr
  5. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoning13 Jan
  6. 2024Generative Verifiers: Reward Modeling as Next-Token Prediction27 Aug

Other references · 13

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  3. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  4. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  5. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 281
  6. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contextscited by 240
  7. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 194
  8. 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoningcited by 134
  9. 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuningcited by 86
  10. 2025J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learningcited by 73
  11. 2025R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learningcited by 65
  12. 2025MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable Step-Level Supervisioncited by 28
1 more
  1. 2025WorldPM: Scaling Human Preference Modelingcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 8.

11 papers.

  1. 2025A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Modelscited by 37
  2. 2025Training Vision-Language Process Reward Models for Test-Time Scaling in Multimodal Reasoning: Key Insights and Lessons Learnedcited by 12
  3. 2025TIM-PRM: Verifying multimodal reasoning with Tool-Integrated PRMcited by 8
  4. 2026PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignmentcited by 5
  5. 2026Grounding the Score: Explicit Visual Premise Verification for Reliable Vision-Language Process Reward Modelscited by 4
  6. 2026PDCR: Perception-Decomposed Confidence Reward for Vision-Language Reasoningcited by 3
  7. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 2
  8. 2026Training Data Efficiency in Multimodal Process Reward Modelscited by 1
  9. 2026LookBack: Where and How to Score LVLM Responses via Visual Reference Usagecited by 0
  10. 2026A Low Grounding Score Is Not an Ungrounded Judge: Identifying the Perceptibility Confound in Multimodal Oversightcited by 0
  11. 2026SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025R-PRM: Reasoning-Driven Process Reward Modeling
  2. 2025Process Reward Models That Think
  3. 2023Let's reward step by step: Step-Level reward model as the Navigators for Reasoning
  4. 2025FreePRM: Training Process Reward Models Without Ground Truth Process Labels
  5. 2026Process Reward Models Meet Planning: Generating Precise and Scalable Datasets for Step-Level Rewards
  6. 2024Free Process Rewards without Process Labels
  7. 2025Benchmarking Multimodal CoT Reward Model Stepwise by Visual Program
  8. 2026ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward