mlsanityData to 7 Oct 2026

LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL

arXiv 2503.07536 ↗First posted 10 Mar 2025cs.CL cs.AIcited by 244

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Enhancing reasoning in Large Multimodal Models (LMMs) faces unique challenges from the complex interplay between visual perception and logical reasoning, particularly in compact 3B-parameter architectures where architectural constraints limit reasoning capacity and modality alignment. While rule-based reinforcement learning (RL) excels in text-only domains, its multimodal extension confronts two critical barriers: (1) data limitations due to ambiguous answers and scarce complex reasoning examples, and (2) degraded foundational reasoning induced by multimodal pretraining. To address these challenges, we propose LMM-R1, a two-stage framework adapting rule-based RL for multimodal reasoning through Foundational Reasoning Enhancement (FRE) followed by Multimodal Generalization Training (MGT). The FRE stage first strengthens reasoning abilities using text-only data with rule-based RL, then the MGT stage generalizes these reasoning capabilities to multimodal domains. Experiments on Qwen2.5-VL-Instruct-3B demonstrate that LMM-R1 achieves 4.83% and 4.5% average improvements over baselines in multimodal and text-only benchmarks, respectively, with a 3.63% gain in complex Football Game tasks. These results validate that text-based reasoning enhancement enables effective multimodal generalization, offering a data-efficient paradigm that bypasses costly high-quality multimodal training data.

Key facts. Cited by 244 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 75 of its 93 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

75 of 93 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Unlocking Multimodal Mathematical Reasoning via Process Reward Model8 Jan
  2. 2024Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models25 Jun
  3. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  4. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning10 Jan
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  6. 2022Super-CLEVR: A Virtual Benchmark to Diagnose Domain Robustness in Visual Reasoning1 Dec
  7. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep
  8. 2022CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoning10 Aug
  9. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar

Other references · 66

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  9. 2025Qwen2.5-VL Technical Reportcited by 5,418
  10. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  12. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
38 more
  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  2. 2023Let's Verify Step by Stepcited by 3,598
  3. 2023Qwen Technical Reportcited by 3,273
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  7. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  8. 2015VQA: Visual Question Answeringcited by 2,489
  9. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  10. 2019Towards VQA Models That Can Readcited by 1,696
  11. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  12. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  13. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  14. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  15. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  16. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  17. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 893
  18. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
  19. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  20. 2016A Diagram Is Worth A Dozen Imagescited by 793
  21. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  22. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  23. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  24. 2023VILA: On Pre-training for Visual Language Modelscited by 686
  25. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  26. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
  27. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  28. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 499
  29. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  30. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  31. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  32. 2021Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoningcited by 376
  33. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 347
  34. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
  35. 2018DVQA: Understanding Data Visualizations via Question Answeringcited by 338
  36. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
  37. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  38. 2019PlotQA: Reasoning over Scientific Plotscited by 317

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 179; 2026: 65.

244 papers.

Sort:

By area:

  1. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
  2. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learningcited by 398
  3. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  4. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 315
  5. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
  6. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 248
  7. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  8. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 209
  9. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 195
  10. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 176
  11. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 166
  12. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learningcited by 163
38 more
  1. 2025Thyme: Think Beyond Imagescited by 160
  2. 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoningcited by 152
  3. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvementcited by 144
  4. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  5. 2025Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokenscited by 120
  6. 2025VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Modelscited by 102
  7. 2025Latent Visual Reasoningcited by 99
  8. 2025Perception-Aware Policy Optimization for Multimodal Reasoningcited by 95
  9. 2025Self-Rewarding Vision-Language Model via Reasoning Decompositioncited by 95
  10. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibilitycited by 89
  11. 2025Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoningcited by 88
  12. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentationcited by 86
  13. 2025More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Modelscited by 84
  14. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learningcited by 83
  15. 2025Scaling RL to Long Videoscited by 83
  16. 2025GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agentscited by 73
  17. 2025KRIS-Bench: Benchmarking Next-Level Intelligent Image Editing Modelscited by 69
  18. 2025R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learningcited by 65
  19. 2025A Survey on (M)LLM-Based GUI Agentscited by 61
  20. 2025TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoningcited by 59
  21. 2025Look-Back: Implicit Visual Re-focusing in MLLM Reasoningcited by 54
  22. 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMscited by 54
  23. 2025SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learningcited by 53
  24. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learningcited by 51
  25. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learningcited by 46
  26. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Rewardcited by 45
  27. 2025Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuningcited by 44
  28. 2025ZeroGUI: Automating Online GUI Learning at Zero Human Costcited by 44
  29. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Modelscited by 43
  30. 2025VisPlay: Self-Evolving Vision-Language Models from Imagescited by 42
  31. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 38
  32. 2025VLA-R1: Enhancing Reasoning in Vision-Language-Action Modelscited by 38
  33. 2025MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and Open Resourcescited by 37
  34. 2025UME-R1: Exploring Reasoning-Driven Generative Multimodal Embeddingscited by 37
  35. 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
  36. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  37. 2025Fast-Slow Thinking GRPO for Large Vision-Language Model Reasoningcited by 34
  38. 2025Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoningcited by 34

50 most cited of 244 All 244 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Advancing Multimodal Reasoning via Reinforcement Learning with Cold Start
  2. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models
  3. 2025Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models
  4. 2025MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning
  5. 2026LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Models
  6. 2025OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe
  7. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization
  8. 2025Boosting Reasoning in Large Multimodal Models via Activation Replay