StructVRM: Aligning Multimodal Reasoning with Structured and Verifiable Reward Models
On the map: LLM reasoning → Multimodal reasoning
also in 2 more
Abstract (arXiv)
Existing Vision-Language Models often struggle with complex, multi-question reasoning tasks where partial correctness is crucial for effective learning. Traditional reward mechanisms, which provide a single binary score for an entire response, are too coarse to guide models through intricate problems with multiple sub-parts. To address this, we introduce StructVRM, a method that aligns multimodal reasoning with Structured and Verifiable Reward Models. At its core is a model-based verifier trained to provide fine-grained, sub-question-level feedback, assessing semantic and mathematical equivalence rather than relying on rigid string matching. This allows for nuanced, partial credit scoring in previously intractable problem formats. Extensive experiments demonstrate the effectiveness of StructVRM. Our trained model, Seed-StructVRM, achieves state-of-the-art performance on six out of twelve public multimodal benchmarks and our newly curated, high-difficulty STEM-Bench. The success of StructVRM validates that training with structured, verifiable rewards is a highly effective approach for advancing the capabilities of multimodal models in complex, real-world reasoning domains.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 42 references that are in the map. First posted on arXiv 7 Aug 2025.
Builds on
33 of 42 references.
Closest ancestors · 12
Same direction, newest first
- 2025MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable Step-Level Supervision19 May
- 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
- 2025Perception-R1: Pioneering Perception Policy with Reinforcement Learning10 Apr
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
- 2025Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme3 Apr
- 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models26 Mar
- 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model7 Mar
- 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
- 2024Polymath: A Challenging Multi-modal Mathematical Reasoning Benchmark6 Oct
- 2024MathScape: Benchmarking Multimodal Large Language Models in Real-World Mathematical Contexts14 Aug
- 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?1 Jul
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
Other references · 21
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
- 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
- 2025Seed1.5-VL Technical Reportcited by 373
- 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 367
- 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
- 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
- 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 301
- 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 285
- 2025RM-R1: Reward Modeling as Reasoningcited by 154
- 2025RLPR: Extrapolating RLVR to General Domains without Verifierscited by 85
9 more
- 2025GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Modelscited by 82
- 2025ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMscited by 46
- 2025R-PRM: Reasoning-Driven Process Reward Modelingcited by 38
- 2024R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?cited by 30
- 2025GRAM: A Generative Foundation Reward Model for Reward Generalizationcited by 25
- 2025SOLIDGEO: Measuring Multimodal Spatial Math Reasoning in Solid Geometrycited by 18
- 2023VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Modelscited by 15
- 2025ReasonGRM: Enhancing Generative Reward Models through Large Reasoning Modelscited by 9
- 2025ChemQuests: A Curated Chemistry Question-Answer Database Extracted from ChemRxiv paperscited by 3
Built on by
Citing papers per year: 2025: 2; 2026: 4.
6 papers.
- 2025MindGPT-4ov: An Enhanced MLLM via a Multi-Stage Post-Training Paradigmcited by 9
- 2026MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learningcited by 5
- 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 5
- 2026Improving Vision-language Models with Perception-centric Process Reward Modelscited by 4
- 2025ResearchPulse: Building Method-Experiment Chains through Multi-Document Scientific Inferencecited by 2
- 2026Smooth Operator: Smooth Verifiable Reward Activates Spatial Reasoning Ability of Vision-Language Modelcited by 2
- 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence31 Aug 2026
- 2026Improving Vision-language Models with Perception-centric Process Reward Models27 Apr 2026
- 2026MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning26 Mar 2026
- 2026Smooth Operator: Smooth Verifiable Reward Activates Spatial Reasoning Ability of Vision-Language Model12 Jan 2026
- 2025MindGPT-4ov: An Enhanced MLLM via a Multi-Stage Post-Training Paradigm2 Dec 2025
- 2025ResearchPulse: Building Method-Experiment Chains through Multi-Document Scientific Inference3 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2025Self-Rewarded Multimodal Coherent Reasoning Across Diverse Visual Domains
- 2025Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Models
- 2026StructReward: Efficient Structured Process Rewards for Self-Correcting Multimodal Reasoning
- 2025VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanism
- 2026V-Zero: Self-Improving Multimodal Reasoning with Zero Annotation
- 2025Closing the Gap: Data-Centric Fine-Tuning of Vision Language Models for the Standardized Exam Questions
- 2026DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification
- 2025RPTS: Tree-Structured Reasoning Process Scoring for Faithful Multimodal Evaluation