Data to 14 Oct 2026

Self-Rewarding Vision-Language Model via Reasoning Decomposition

arXiv 2508.19652 ↗First posted 27 Aug 2025cs.CVcited by 97

On the map: LLM reasoning → Multimodal reasoning

also in 2 more

Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Synthetic data and self-improvement for LLMs → Self-improving MLLMs and synthetic data

Abstract (arXiv)

Vision-Language Models (VLMs) often suffer from visual hallucinations: generating things that are not consistent with visual inputs and language shortcuts, where they skip the visual part and just rely on text priors. These issues arise because most post training methods for VLMs rely on simple verifiable answer matching and supervise only final outputs, leaving intermediate visual reasoning without explicit guidance. As a result, VLMs receive sparse visual signals and often learn to prioritize language based reasoning over visual perception. We introduce Vision SR1, a three stage self rewarding reinforcement learning method that improves visual reasoning without relying on external visual supervision. Vision SR1 decomposes VLM reasoning into two components: visual reasoning and language reasoning, where the model is first prompted to produce self-contained visual descriptions sufficient to answer the question without referring back to the input image, before jointly optimizing both visual and language reasoning through our multi reward loss objective. To validate this self containment, the same VLM model is reprompted to perform language reasoning using only the generated visual reasoning as input to compute visual reward. The final reward is computed through a decoupled reward-advantage framework, where visual reward and language reasoning reward each have their advantages calculated separately. Our experiments show that Vision SR1 improves visual reasoning, mitigates visual hallucinations, and reduces reliance on language shortcuts across diverse vision language tasks, while being more efficient than methods that rely on external visual reward models, which require additional GPUs to host. In contrast, Vision SR1 introduces no extra GPU overhead beyond that of standard training.

Key facts. Cited by 97 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 46 of its 61 references that are in the map. First posted on arXiv 27 Aug 2025.

Builds on

46 of 61 references.

Closest ancestors · 9

Same direction, newest first

Other references · 37

2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
2022Scaling Laws for Reward Model Overoptimizationcited by 1,095
2024Self-Rewarding Language Modelscited by 681
2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 512
2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 445
2024A Survey on Hallucination in Large Vision-Language Modelscited by 357
2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 301

25 more

2025R-Zero: Self-Evolving Reasoning LLM from Zero Datacited by 228
2025Learning to Reason without External Rewardscited by 210
2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 209
2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 183
2024RLAIF-V: Open-Source AI Feedback Leads to Super GPT-4V Trustworthinesscited by 131
2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Modelscited by 114
2025Reward Shaping to Mitigate Reward Hacking in RLHFcited by 88
2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 86
2024Calibrated Self-Rewarding Vision Language Modelscited by 78
2025Are Reasoning Models More Prone to Hallucination?cited by 64
2025Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Modelscited by 62
2025Exploring the Limit of Outcome Reward for Learning Mathematical Reasoningcited by 52
2025Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systemscited by 51
2025VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understandingcited by 32
2022Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQAcited by 30
2024Wings: Learning Multimodal LLMs without Text-only Forgettingcited by 23
2025Learning to Reason via Mixture-of-Thought for Logical Reasoningcited by 20
2025Enhancing Rating-Based Reinforcement Learning to Effectively Leverage Feedback from Large Vision-Language Modelscited by 15
2025Post-Training Large Language Models via Reinforcement Learning from Self-Feedbackcited by 15
2025VisNumBench: Evaluating Number Sense of Multimodal Large Language Modelscited by 14
2025Semantically-Aware Rewards for Open-Ended R1 Training in Free-Form Generationcited by 11
2025See or Recall: A Sanity Check for the Role of Vision in Solving Visualization Question Answer Tasks with Multimodal LLMscited by 7
2025Attention-Based Reward Shaping for Sparse and Delayed Rewardscited by 5
2025Multimodal Reference Visual Groundingcited by 3
2025Reward Generation via Large Vision-Language Model in Offline Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 32; 2026: 65.

97 papers.

Sort:

By area:

2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Modelscited by 90
2025Parallel-R1: Towards Parallel Thinking via Reinforcement Learningcited by 61
2026Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skillscited by 55
2025VisPlay: Self-Evolving Vision-Language Models from Imagescited by 47
2025Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variationcited by 45
2026Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challengescited by 33
2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoningcited by 31
2026Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Gradingcited by 31
2025CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Modelscited by 29
2025TruthRL: Incentivizing Truthful LLMs via Reinforcement Learningcited by 27
2025EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewardscited by 27

38 more

2025ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-Language Models through Agentic Data Synthesiscited by 26
2025V-Thinker: Interactive Thinking with Imagescited by 26
2026Visual-Advantage On-Policy Distillation for Vision-Language Modelscited by 24
2026MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Datacited by 21
2025Perceptual-Evidence Anchored Reinforced Learning for Multimodal Reasoningcited by 20
2026Recursive Synthesis for Long-Horizon Terminal Taskscited by 19
2025A co-evolving agentic AI system for medical imaging analysiscited by 17
2025Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMscited by 17
2026Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Taskscited by 15
2025DeepSketcher: Internalizing Visual Manipulation for Multimodal Reasoningcited by 14
2025First Frame Is the Place to Go for Video Content Customizationcited by 12
2025DualVLA: Building a Generalizable Embodied Agent via Partial Decoupling of Reasoning and Actioncited by 12
2026SAMTok: Representing Any Mask with Two Wordscited by 11
2026Save the Good Prefix: Precise Error Penalization via Process-Supervised RL to Enhance LLM Reasoningcited by 11
2025DeFacto: Counterfactual Thinking with Images for Enforcing Evidence-Grounded and Faithful Reasoningcited by 10
2026Decomposed On-Policy Distillation for Vision-Language Reasoning: Steering Gradients for Visual Groundingcited by 10
2026Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoningcited by 9
2025VIDEOP2R: Video Understanding from Perception to Reasoningcited by 8
2025Stable and Efficient Single-Rollout RL for Multimodal Reasoningcited by 8
2026RelayLLM: Efficient Reasoning via Collaborative Decodingcited by 8
2025ReAG: Reasoning-Augmented Generation for Knowledge-based Visual Question Answeringcited by 7
2026iReasoner: Trajectory-Aware Intrinsic Reasoning Supervision for Self-Evolving Large Multimodal Modelscited by 7
2026RP-OPSD: Resolution-Privileged On-Policy Self-Distillation for Multimodal Large Language Modelscited by 6
2025Dual-Uncertainty Guided Policy Learning for Multimodal Reasoningcited by 5
2025MASS: Motion-Aware Spatial-Temporal Grounding for Physics Reasoning and Comprehension in Vision-Language Modelscited by 5
2025Be My Eyes: Extending Large Language Models to New Modalities Through Multi-Agent Collaborationcited by 5
2025VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMscited by 5
2026ContextRL: Enhancing MLLM's Knowledge Discovery Efficiency with Context-Augmented RLcited by 5
2026SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environmentscited by 5
2026Learning Adaptive Reasoning Paths for Efficient Visual Reasoningcited by 5
2025VTPerception-R1: Enhancing Multimodal Reasoning via Explicit Visual and Textual Perceptual Groundingcited by 4
2026RegionReasoner: Region-Grounded Multi-Round Visual Reasoningcited by 4
2026EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformationscited by 4
2026Seeing Before Reasoning: Decoupling Perception and Reasoning for Shortcut-Resilient Multimodal On-Policy Self-Distillationcited by 4
2025ViLaCD-R1: A Vision-Language Framework for Semantic Change Detection in Remote Sensingcited by 3
2026DR2Seg: Decomposed Two-Stage Rollouts for Efficient Reasoning Segmentation in Multimodal Large Language Modelscited by 3
2026HART: High-Resolution Annotation-Free Reasoning Technique through a Closed-loop Frameworkcited by 3
2026VERTIGO: Visual Preference Optimization for Cinematic Camera Trajectory Generationcited by 3

50 most cited of 97 All 97 (JSON)

Similar, not cited

Close in wording, no citation either way