mlsanityData to 7 Oct 2026

VisualPRM: An Effective Process Reward Model for Multimodal Reasoning

arXiv 2503.10291 ↗First posted 13 Mar 2025cs.CV cs.CLcited by 134

On the map: LLM reasoning → Multimodal reasoning

also in 2 more
Abstract (arXiv)

We introduce VisualPRM, an advanced multimodal Process Reward Model (PRM) with 8B parameters, which improves the reasoning abilities of existing Multimodal Large Language Models (MLLMs) across different model scales and families with Best-of-N (BoN) evaluation strategies. Specifically, our model improves the reasoning performance of three types of MLLMs and four different model scales. Even when applied to the highly capable InternVL2.5-78B, it achieves a 5.9-point improvement across seven multimodal reasoning benchmarks. Experimental results show that our model exhibits superior performance compared to Outcome Reward Models and Self-Consistency during BoN evaluation. To facilitate the training of multimodal PRMs, we construct a multimodal process supervision dataset VisualPRM400K using an automated data pipeline. For the evaluation of multimodal PRMs, we propose VisualProcessBench, a benchmark with human-annotated step-wise correctness labels, to measure the abilities of PRMs to detect erroneous steps in multimodal reasoning tasks. We hope that our work can inspire more future research and contribute to the development of MLLMs. Our model, data, and benchmark are released in https://internvl.github.io/blog/2025-03-13-VisualPRM/.

Key facts. Cited by 134 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 84 of its 98 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

84 of 98 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  2. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization15 Nov
  3. 2024DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models29 Oct
  4. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul
  5. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?1 Jul
  6. 2024M3CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought26 May
  7. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  8. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  9. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep
  10. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar

Other references · 74

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  9. 2023Visual Instruction Tuningcited by 7,792
  10. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  11. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
38 more
  1. 2025Qwen2.5-VL Technical Reportcited by 5,418
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2024GPT-4o System Cardcited by 4,467
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  7. 2023Let's Verify Step by Stepcited by 3,598
  8. 2023Qwen Technical Reportcited by 3,273
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  10. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  11. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  12. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  13. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  14. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  15. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  16. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,928
  17. 2019Towards VQA Models That Can Readcited by 1,696
  18. 2025s1: Simple test-time scalingcited by 1,311
  19. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  20. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  21. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  22. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  23. 2016A Diagram Is Worth A Dozen Imagescited by 793
  24. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  25. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  26. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  27. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  28. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  29. 2021InfographicVQAcited by 430
  30. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  31. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 402
  32. 2024InternLM2 Technical Reportcited by 391
  33. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 376
  34. 2024Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Expertscited by 369
  35. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  36. 2019Scene Text Visual Question Answeringcited by 327
  37. 2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documentscited by 316
  38. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 277

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 82; 2026: 52.

134 papers.

Sort:

By area:

  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  2. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  3. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  4. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 166
  5. 2025RewardBench 2: Advancing Reward Model Evaluationcited by 136
  6. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 122
  7. 2025VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Modelscited by 102
  8. 2025Latent Visual Reasoningcited by 99
  9. 2025GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoningcited by 94
  10. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
  11. 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 84
  12. 2025AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progresscited by 69
38 more
  1. 2025VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perceptioncited by 60
  2. 2025Reinforcement Learning from Human Feedbackcited by 53
  3. 2025SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Rewardcited by 45
  4. 2025EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversitycited by 41
  5. 2025Reinforced Visual Perception with Toolscited by 38
  6. 2025A Survey of Process Reward Models: From Outcome Signals to Process Supervisions for Large Language Modelscited by 37
  7. 2025LLaVA-Critic-R1: Your Critic Model is Secretly a Strong Policy Modelcited by 34
  8. 2025UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agentscited by 33
  9. 2025UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generationcited by 32
  10. 2025Skywork-VL Reward: An Effective Reward Model for Multimodal Understanding and Reasoningcited by 30
  11. 2025Visual Document Understanding and Reasoning: A Multi-Agent Collaboration Framework with Agent-Wise Adaptive Test-Time Scalingcited by 30
  12. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoningcited by 28
  13. 2025BusterX++: Towards Unified Cross-Modal AI-Generated Content Detection and Explanation with MLLMcited by 28
  14. 2025VL-Cogito: Progressive Curriculum Reinforcement Learning for Advanced Multimodal Reasoningcited by 26
  15. 2025OCR-Reasoning Benchmark: Unveiling the True Capabilities of MLLMs in Complex Text-Rich Image Reasoningcited by 25
  16. 2025EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewardscited by 24
  17. 2025WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learningcited by 21
  18. 2025Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmarkcited by 20
  19. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
  20. 2025Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsingcited by 17
  21. 2025ViLBench: A Suite for Vision-Language Process Reward Modelingcited by 16
  22. 2025Improving Medical Reasoning with Curriculum-Aware Reinforcement Learningcited by 16
  23. 2025Athena: Enhancing Multimodal Reasoning with Data-efficient Process Reward Modelscited by 16
  24. 2025Chiron-o1: Igniting Multimodal Large Language Models towards Generalizable Medical Reasoning via Mentor-Intern Collaborative Searchcited by 16
  25. 2025BaseReward: A Strong Baseline for Multimodal Reward Modelcited by 15
  26. 2025Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewardscited by 14
  27. 2025GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinkingcited by 14
  28. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 13
  29. 2025Unveiling the Compositional Ability Gap in Vision-Language Reasoning Modelcited by 13
  30. 2025GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoningcited by 13
  31. 2025Is PRM Necessary? Problem-Solving RL Implicitly Induces PRM Capability in LLMscited by 12
  32. 2025Training Vision-Language Process Reward Models for Test-Time Scaling in Multimodal Reasoning: Key Insights and Lessons Learnedcited by 12
  33. 2025BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Datasetcited by 11
  34. 2025VRPRM: Process Reward Modeling via Visual Reasoningcited by 11
  35. 2025AutoRubric: Rubric-Based Generative Rewards for Faithful Multimodal Reasoningcited by 11
  36. 2026Thinking in Uncertainty: Mitigating Hallucinations in MLRMs with Latent Entropy-Aware Decodingcited by 11
  37. 2025Advancing Reasoning in Diffusion Language Models with Denoising Process Rewardscited by 9
  38. 2025Trust but Verify! A Survey on Verification Design for Test-time Scalingcited by 8

50 most cited of 134 All 134 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Unlocking Multimodal Mathematical Reasoning via Process Reward Model
  2. 2025MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process Errors Identification
  3. 2025DreamPRM: Domain-Reweighted Process Reward Model for Multimodal Reasoning
  4. 2025MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable Step-Level Supervision
  5. 2025Self-Rewarded Multimodal Coherent Reasoning Across Diverse Visual Domains
  6. 2025PRM-BAS: Enhancing Multimodal Reasoning through PRM-guided Beam Annealing Search
  7. 2025RPTS: Tree-Structured Reasoning Process Scoring for Faithful Multimodal Evaluation
  8. 2025VLRMBench: A Comprehensive and Challenging Benchmark for Vision-Language Reward Models