mlsanityData to 14 Oct 2026

Benchmarking Multimodal CoT Reward Model Stepwise by Visual Program

arXiv 2504.06606 ↗First posted 9 Apr 2025cs.CVcited by 10

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Recent advancements in reward signal usage for Large Language Models (LLMs) are remarkable. However, significant challenges exist when transitioning reward signal to the multimodal domain, including labor-intensive annotations, over-reliance on one-step rewards, and inadequate evaluation. To address these issues, we propose SVIP, a novel approach to train a step-level multi-dimensional Chain-of-Thought~(CoT) reward model automatically. It generates code for solving visual tasks and transforms the analysis of code blocks into the evaluation of CoT step as training samples. Then, we train SVIP-Reward model using a multi-head attention mechanism called TriAtt-CoT. The advantages of SVIP-Reward are evident throughout the entire process of MLLM. We also introduce a benchmark for CoT reward model training and testing. Experimental results demonstrate that SVIP-Reward improves MLLM performance across training and inference-time scaling, yielding better results on benchmarks while reducing hallucinations and enhancing reasoning ability.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 45 references that are in the map. First posted on arXiv 9 Apr 2025.

Builds on

38 of 45 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model21 Jan
  2. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  3. 2024STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Training29 Nov
  4. 2024M3CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought26 May
  5. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar
  6. 2024PropTest: Automatic Property Testing for Improved Visual Programming25 Mar
  7. 2024Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning18 Feb
  8. 2024CoCoT: Contrastive Chain-of-Thought Prompting for Large Multimodal Models with Multiple Image Inputs5 Jan
  9. 2023Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models5 Dec
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  11. 2023ViperGPT: Visual Inference via Python Execution for Reasoning14 Mar
  12. 2022Visual Programming: Compositional visual reasoning without training18 Nov

Other references · 26

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  5. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  6. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  7. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  8. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  9. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  10. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  11. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  12. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
14 more
  1. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  2. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 565
  3. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 356
  4. 2024MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGIcited by 193
  5. 2025PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Modelscited by 99
  6. 2024VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Modelscited by 86
  7. 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 77
  8. 2022Fine-Grained Semantically Aligned Vision-Language Pre-Trainingcited by 68
  9. 2024MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?cited by 66
  10. 2024RMB: Comprehensively Benchmarking Reward Models in LLM Alignmentcited by 61
  11. 2024WorldGPT: Empowering LLM as Multimodal World Modelcited by 42
  12. 2023Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Groundingcited by 19
  13. 2023De-fine: Decomposing and Refining Visual Programs with Auto-Feedbackcited by 10
  14. 2024Fact :Teaching MLLMs with Faithful, Concise and Transferable Rationalescited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 3.

10 papers.

  1. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learningcited by 53
  2. 2025Evaluating MLLMs with Multimodal Multi-image Reasoning Benchmarkcited by 21
  3. 2025Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewardscited by 14
  4. 2025What Limits Virtual Agent Application? OmniBench: A Scalable Multi-Dimensional Benchmark for Essential Virtual Agent Capabilitiescited by 8
  5. 2026Visual Preference Optimization with Rubric Rewardscited by 5
  6. 2025Enhancing LLMs' Reasoning-Intensive Multimedia Search Capabilities through Fine-Tuning and Reinforcement Learningcited by 3
  7. 2026Benchmarking PhD-Level Coding in 3D Geometric Computer Visioncited by 2
  8. 2025AgroCoT: A Chain-of-Thought Benchmark for Evaluating Reasoning in Vision-Language Models for Agriculturecited by 1
  9. 2025Understanding Chain-of-Thought in Large Language Models via Topological Data Analysiscited by 1
  10. 2026GRACE: Gradient-aligned Reasoning Data Curation for Efficient Post-trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Models
  2. 2025VLRMBench: A Comprehensive and Challenging Benchmark for Vision-Language Reward Models
  3. 2025ViLBench: A Suite for Vision-Language Process Reward Modeling
  4. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
  5. 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning
  6. 2026Attention-guided Fine-tuning of Multimodal Large Language Models Improves Chain-of-Thought Reasoning
  7. 2025Training Vision-Language Process Reward Models for Test-Time Scaling in Multimodal Reasoning: Key Insights and Lessons Learned
  8. 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoning