mlsanityData to 14 Oct 2026

R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model

arXiv 2503.05132 ↗First posted 7 Mar 2025cs.AI cs.CV cs.LGcited by 165

On the map: LLM reasoning → Multimodal reasoning

Abstract (arXiv)

Recently DeepSeek R1 demonstrated how reinforcement learning with simple rule-based incentives can enable autonomous development of complex reasoning in large language models, characterized by the "aha moment", in which the model manifest self-reflection and increased response length during training. However, attempts to extend this success to multimodal reasoning often failed to reproduce these key characteristics. In this report, we present the first successful replication of these emergent characteristics for multimodal reasoning on only a non-SFT 2B model. Starting with Qwen2-VL-2B and applying reinforcement learning directly on the SAT dataset, our model achieves 59.47% accuracy on CVBench, outperforming the base model by approximately ~30% and exceeding both SFT setting by ~2%. In addition, we share our failed attempts and insights in attempting to achieve R1-like reasoning using RL with instruct models. aiming to shed light on the challenges involved. Our key observations include: (1) applying RL on instruct model often results in trivial reasoning trajectories, and (2) naive length reward are ineffective in eliciting reasoning capabilities. The project code is available at https://github.com/turningpoint-ai/VisualThinker-R1-Zero

Key facts. Cited by 165 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 7 of its 13 references that are in the map. First posted on arXiv 7 Mar 2025.

Builds on

7 of 13 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025LlamaV-o1: Rethinking Step-by-step Visual Reasoning in LLMs10 Jan
  2. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
  3. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models13 Jun

Other references · 4

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  4. 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 114

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 124; 2026: 41.

165 papers.

Sort:

By area:

  1. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 526
  2. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 410
  3. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  4. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 255
  5. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 212
  6. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  7. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 168
  8. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
  9. 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoningcited by 150
  10. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 132
  11. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 124
  12. 2025Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Searchcited by 120
38 more
  1. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Modelscited by 90
  2. 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Usecited by 90
  3. 2025Visual Planning: Let's Think Only with Imagescited by 85
  4. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learningcited by 84
  5. 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 74
  6. 2025Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answeringcited by 65
  7. 2025VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perceptioncited by 65
  8. 2025TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoningcited by 62
  9. 2025VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agentscited by 62
  10. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 55
  11. 2025Improved Visual-Spatial Reasoning via R1-Zero-Like Trainingcited by 51
  12. 2025Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuningcited by 44
  13. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 44
  14. 2025ZeroGUI: Automating Online GUI Learning at Zero Human Costcited by 44
  15. 2025AgentCPM-GUI: Building Mobile-Use Agents with Reinforcement Fine-Tuningcited by 44
  16. 2025Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokenscited by 41
  17. 2025Advancing Multimodal Reasoning via Reinforcement Learning with Cold Startcited by 41
  18. 2025EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversitycited by 41
  19. 2025MMedAgent-RL: Optimizing Multi-Agent Collaboration for Multimodal Medical Reasoningcited by 40
  20. 2025Reinforced Visual Perception with Toolscited by 38
  21. 2025UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddingscited by 38
  22. 2025Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!cited by 37
  23. 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
  24. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecyclecited by 35
  25. 2025First SFT, Second RL, Third UPT: Continual Improving Multi-Modal LLM Reasoning via Unsupervised Post-Trainingcited by 34
  26. 2025MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMscited by 33
  27. 2025ChestX-Reasoner: Advancing Radiology Foundation Models with Reasoning through Step-by-Step Verificationcited by 31
  28. 2025X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domainscited by 31
  29. 2025Reinforcing Video Reasoning with Focused Thinkingcited by 31
  30. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  31. 2025MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPOcited by 30
  32. 2025MoDoMoDo: Multi-Domain Data Mixtures for Multimodal LLM Reinforcement Learningcited by 30
  33. 2025VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinkingcited by 30
  34. 2025EchoInk-R1: Exploring Audio-Visual Reasoning in Multimodal LLMs via Reinforcement Learningcited by 29
  35. 2025Reinforcement Learning Tuning for VideoLLMs: Reward Design and Data Efficiencycited by 26
  36. 2025OCR-Reasoning Benchmark: Unveiling the True Capabilities of MLLMs in Complex Text-Rich Image Reasoningcited by 25
  37. 2025VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuningcited by 24
  38. 2025Simple o3: Towards Interleaved Vision-Language Reasoningcited by 24

50 most cited of 165 All 165 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
  2. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning
  3. 2025Understanding R1-Zero-Like Training: A Critical Perspective
  4. 2025Observe-R1: Unlocking Reasoning Abilities of MLLMs with Dynamic Progressive Reinforcement Learning
  5. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Models
  6. 2025An Empirical Study on Eliciting and Improving R1-like Reasoning Models
  7. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild
  8. 2025Metis-RISE: RL Incentivizes and SFT Enhances Multimodal Reasoning Model Learning