mlsanityData to 7 Oct 2026

VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model

arXiv 2504.07615 ↗First posted 10 Apr 2025cs.CV cs.CLcited by 515

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Recently DeepSeek R1 has shown that reinforcement learning (RL) can substantially improve the reasoning capabilities of Large Language Models (LLMs) through a simple yet effective design. The core of R1 lies in its rule-based reward formulation, which leverages tasks with deterministic ground-truth answers to enable precise and stable reward computation. In the visual domain, we similarly observe that a wide range of visual understanding tasks are inherently equipped with well-defined ground-truth annotations. This property makes them naturally compatible with rule-based reward mechanisms. Motivated by this observation, we investigate the extension of R1-style reinforcement learning to Vision-Language Models (VLMs), aiming to enhance their visual reasoning capabilities. To this end, we develop VLM-R1, a dedicated framework designed to harness RL for improving VLMs' performance on general vision-language tasks. Using this framework, we further explore the feasibility of applying RL to visual domain. Experimental results indicate that the RL-based model not only delivers competitive performance on visual understanding tasks but also surpasses Supervised Fine-Tuning (SFT) in generalization ability. Furthermore, we conduct comprehensive ablation studies that uncover a series of noteworthy insights, including the presence of reward hacking in object detection, the emergence of the "OD aha moment", the impact of training data quality, and the scaling behavior of RL across different model sizes. Through these analyses, we aim to deepen the understanding of how reinforcement learning enhances the capabilities of vision-language models, and we hope our findings and open-source contributions will support continued progress in the vision-language RL community. Our code and model are available at https://github.com/om-ai-lab/VLM-R1

Key facts. Cited by 515 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 48 of its 58 references that are in the map. First posted on arXiv 10 Apr 2025.

Builds on

48 of 58 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization13 Mar
  2. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  3. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning10 Mar
  4. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  5. 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model7 Mar
  6. 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
  7. 2025InternLM-XComposer2.5-Reward: A Simple Yet Effective Multi-Modal Reward Model21 Jan
  8. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  9. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  10. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  11. 2023LISA: Reasoning Segmentation via Large Language Model1 Aug

Other references · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2017Proximal Policy Optimization Algorithmscited by 14,441
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  7. 2023Visual Instruction Tuningcited by 7,792
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  10. 2025Qwen2.5-VL Technical Reportcited by 5,418
  11. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  12. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
25 more
  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  6. 2016Concrete Problems in AI Safetycited by 1,582
  7. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  8. 2016Modeling Context in Referring Expressionscited by 1,054
  9. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  10. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  11. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  12. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  13. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  14. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 655
  15. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  16. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 322
  17. 2023Grounding Language Models to Images for Multimodal Inputs and Outputscited by 129
  18. 2024Grounding DINO 1.5: Advance the "Edge" of Open-Set Object Detectioncited by 122
  19. 2024Language Models Learn to Mislead Humans via RLHFcited by 118
  20. 2023LLMs as Narcissistic Evaluators: When Ego Inflates Evaluation Scorescited by 88
  21. 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 70
  22. 2023Described Object Detection: Liberating Object Detection with Flexible Expressionscited by 40
  23. 2024Spontaneous Reward Hacking in Iterative Self-Refinementcited by 24
  24. 2023How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detectioncited by 15
  25. 2022OmDet: Large-scale vision-language multi-dataset pre-training with multimodal detection networkcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 301; 2026: 214.

515 papers.

Sort:

By area:

  1. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  2. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
  3. 2025ToolRL: Reward is All Tool Learning Needscited by 342
  4. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
  5. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  6. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  7. 2025Thyme: Think Beyond Imagescited by 160
  8. 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 125
  9. 2025GRIT: Teaching MLLMs to Think with Imagescited by 121
  10. 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 107
  11. 2025VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Modelscited by 102
  12. 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 95
38 more
  1. 2025Self-Rewarding Vision-Language Model via Reasoning Decompositioncited by 95
  2. 2025UI-Venus Technical Report: Building High-performance UI Agents with RFTcited by 90
  3. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 89
  4. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 88
  5. 2025Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learningcited by 86
  6. 2025Visual Planning: Let's Think Only with Imagescited by 85
  7. 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 84
  8. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learningcited by 83
  9. 2025SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Modelscited by 83
  10. 2025Grounded Reinforcement Learning for Visual Reasoningcited by 80
  11. 2025Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learningcited by 79
  12. 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 73
  13. 2025VGR: Visual Grounded Reasoningcited by 70
  14. 2025SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experiencecited by 68
  15. 2025Nemotron-Research-Tool-N1: Exploring Tool-Using Language Models with Reinforced Reasoningcited by 67
  16. 2025Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodologycited by 67
  17. 2025R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learningcited by 65
  18. 2025GUI-G2: Gaussian Reward Modeling for GUI Groundingcited by 60
  19. 2025VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agentscited by 60
  20. 2025TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoningcited by 59
  21. 2025LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Callingcited by 58
  22. 2025DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPOcited by 57
  23. 2025SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learningcited by 53
  24. 2025Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Modelscited by 52
  25. 2025Uni-cot: Towards Unified Chain-of-Thought Reasoning Across Text and Visioncited by 52
  26. 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 52
  27. 2025HumanOmniV2: From Understanding to Omni-Modal Reasoning with Contextcited by 50
  28. 2025Graph-R1: Towards Agentic GraphRAG Framework via End-to-end Reinforcement Learningcited by 50
  29. 2025Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operationscited by 46
  30. 2025SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Rewardcited by 45
  31. 2025ZeroGUI: Automating Online GUI Learning at Zero Human Costcited by 44
  32. 2025Seg-R1: Segmentation Can Be Surprisingly Simple with Reinforcement Learningcited by 43
  33. 2025ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learningcited by 41
  34. 2025VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learningcited by 41
  35. 2025JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agentcited by 39
  36. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 38
  37. 2025Patho-R1: A Multimodal Reinforcement Learning-Based Pathology Expert Reasonercited by 38
  38. 2025Omni-R1: Reinforcement Learning for Omnimodal Reasoning via Two-System Collaborationcited by 38

50 most cited of 515 All 515 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme
  2. 2024Code as Reward: Empowering Reinforcement Learning with VLMs
  3. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback
  4. 2025Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning
  5. 2023Vision-Language Models as a Source of Rewards
  6. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning
  7. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models
  8. 2025VL-GenRM: Enhancing Vision-Language Verification via Vision Experts and Iterative Training