mlsanityData to 14 Oct 2026

Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models

arXiv 2504.21277 ↗First posted 30 Apr 2025cs.AIcited by 44

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

The application of reinforcement learning (RL) to enhance the reasoning capabilities of Multimodal Large Language Models (MLLMs) constitutes a rapidly advancing research area. While MLLMs extend Large Language Models (LLMs) to handle diverse modalities such as vision, audio, and video, enabling robust reasoning across multimodal inputs remains challenging. This paper provides a systematic review of recent advances in RL-based reasoning for MLLMs, covering key algorithmic designs, reward mechanism innovations, and practical applications. We highlight two main RL paradigms, value-model-free and value-model-based methods, and analyze how RL enhances reasoning abilities by optimizing reasoning trajectories and aligning multimodal information. Additionally, we provide an extensive overview of benchmark datasets, evaluation protocols, and current limitations, and propose future research directions to address challenges such as sparse rewards, inefficient cross-modal reasoning, and real-world deployment constraints. Our goal is to provide a comprehensive and structured guide to RL-based multimodal reasoning.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 116 of its 128 references that are in the map. First posted on arXiv 30 Apr 2025.

Builds on

116 of 128 references.

Closest ancestors · 35

Same direction, newest first

  1. 2025Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning23 Apr
  2. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation17 Apr
  3. 2025TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoning13 Apr
  4. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models10 Apr
  5. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
  6. 2025VCR-Bench: A Comprehensive Evaluation Framework for Video Chain-of-Thought Reasoning10 Apr
  7. 2025Perception-R1: Pioneering Perception Policy with Reinforcement Learning10 Apr
  8. 2025SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement10 Apr
  9. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning9 Apr
  10. 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
  11. 2025OThink-MR1: Stimulating multimodal generalized reasoning capabilities via dynamic reinforcement learning20 Mar
  12. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey16 Mar
23 more
  1. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization13 Mar
  2. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  3. 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement9 Mar
  4. 2025GFlowVLM: Enhancing Multi-step Reasoning in Vision-Language Models with Generative Flow Networks9 Mar
  5. 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model7 Mar
  6. 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
  7. 2025MV-MATH: Evaluating Multimodal Math Reasoning in Multi-Visual Contexts28 Feb
  8. 2025Investigating Inference-time Scaling for Chain of Multi-modal Thought: A Preliminary Study17 Feb
  9. 2025MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency13 Feb
  10. 2025The Jumping Reasoning Curve? Tracking the Evolution of Reasoning Performance in GPT-[n] and o-[n] Models on Multimodal Puzzles3 Feb
  11. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark9 Jan
  12. 2025InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflection8 Jan
  13. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
  14. 2024Improve Vision Language Model Chain-of-thought Reasoning21 Oct
  15. 2024MLLM-CompBench: A Comparative Reasoning Benchmark for Multimodal LLMs23 Jul
  16. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts6 Jul
  17. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?1 Jul
  18. 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation14 Jun
  19. 2024Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition7 May
  20. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset22 Feb
  21. 2023ChartBench: A Benchmark for Complex Visual Reasoning in Charts26 Dec
  22. 2023See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoning12 Jan
  23. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering20 Sep

Other references · 81

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2023Segment Anythingcited by 8,234
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  9. 2025Qwen2.5-VL Technical Reportcited by 5,587
  10. 2024Qwen2.5 Technical Reportcited by 4,942
  11. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  12. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
38 more
  1. 2023Let's Verify Step by Stepcited by 3,789
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  4. 2023Qwen Technical Reportcited by 3,344
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  7. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  8. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  9. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  10. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  11. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  12. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
  13. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  14. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  15. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  16. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  17. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  18. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  19. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  20. 2025Qwen2.5-Omni Technical Reportcited by 736
  21. 2018Learning Dexterous In-Hand Manipulationcited by 687
  22. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  23. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 615
  24. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  25. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  26. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  27. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 527
  28. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  29. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 461
  30. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  31. 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
  32. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 301
  33. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 295
  34. 2025VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Taskscited by 285
  35. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 244
  36. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 241
  37. 2024CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMscited by 234
  38. 2024Chain-of-Thought Reasoning Without Promptingcited by 210

50 most cited of 81 All 81 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 21; 2026: 23.

44 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  2. 2025OneThinker: All-in-one Reasoning Model for Image and Videocited by 56
  3. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  4. 2025Reinforcement Learning Finetunes Small Subnetworks in Large Language Modelscited by 49
  5. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 43
  6. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarkscited by 33
  7. 2025Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Modelscited by 28
  8. 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Modelscited by 26
  9. 2025WeThink: Toward General-purpose Vision-Language Reasoning via Reinforcement Learningcited by 21
  10. 2025Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Datacited by 21
  11. 2025RULE: Reinforcement UnLEarning Achieves Forget-Retain Pareto Optimalitycited by 16
  12. 2025Co-Reinforcement Learning for Unified Multimodal Understanding and Generationcited by 15
32 more
  1. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 14
  2. 2025VisualSphinx: Large-Scale Synthetic Vision Logic Puzzles for RLcited by 12
  3. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  4. 2025Perception-Consistency Multimodal Large Language Models Reasoning via Caption-Regularized Policy Optimizationcited by 9
  5. 2026CamReasoner: Reinforcing Camera Movement Understanding via Structured Spatial Reasoningcited by 9
  6. 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Modelscited by 8
  7. 2026From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoningcited by 8
  8. 2025M2IO-R1: An Efficient RL-Enhanced Reasoning Framework for Multimodal Retrieval Augmented Multimodal Generationcited by 7
  9. 2026EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Modelscited by 5
  10. 2026Learning Adaptive Reasoning Paths for Efficient Visual Reasoningcited by 5
  11. 2025ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replaycited by 4
  12. 2026What does RL improve for Visual Reasoning? A Frankenstein-Style Analysiscited by 4
  13. 2026Implicit Turn-Wise Policy Optimization for Proactive User-LLM Interactioncited by 4
  14. 2026StruVis: Enhancing Reasoning-based Text-to-Image Generation via Thinking with Structured Visioncited by 3
  15. 2026SlowBA: An efficiency backdoor attack towards VLM-based GUI agentscited by 3
  16. 2026ArogyaSutra: A Multi-Agent Framework for Multimodal Medical Reasoning in Indic Languagescited by 3
  17. 2026From Failure to Feedback: Group Revision Unlocks Hard Cases in Object-Level Groundingcited by 2
  18. 2025FedCoT: Communication-Efficient Federated Reasoning Enhancement for Large Language Modelscited by 1
  19. 2026SketchThinker-R1: Towards Efficient Sketch-Style Reasoning in Large Multimodal Modelscited by 1
  20. 2026OmniJigsaw: Enhancing Omni-Modal Reasoning via Modality-Orchestrated Reorderingcited by 1
  21. 2026Rethinking Agentic Reinforcement Learning In Large Language Modelscited by 1
  22. 2026AnE: Pushing the Reasoning Frontier of Multimodal LLMs via Anchor Evolutioncited by 1
  23. 2025Boosting RL-Based Visual Reasoning with Selective Adversarial Entropy Interventioncited by 0
  24. 2026Context-Aware RL for Agentic and Multimodal LLMscited by 0
  25. 2026ThinkDeception: A Progressive Reinforcement Learning Framework for Interpretable Multimodal Deception Detectioncited by 0
  26. 2026Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learningcited by 0
  27. 2026Glance, Scrutinize, and Think: Advancing Video Anomaly Detection from Training-Free to Agentic Reasoningcited by 0
  28. 2026Reasoning for Social Audio-Visual Question Answering: Where Do We Stand?cited by 0
  29. 2026WM-R1: Training GUI Agents to Reason and leverage World Models with Reinforcement Learningcited by 0
  30. 2026Active Spatial Inspection for Effective and Efficient Embodied Explorationcited by 0
  31. 2026Taming CoT Obfuscation in VLMs: From Mechanistic Evidence to Activation Enforcementcited by 0
  32. 2026Look Where You Say You're Looking: Self-Grounded Attention for Visual Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Observe-R1: Unlocking Reasoning Abilities of MLLMs with Dynamic Progressive Reinforcement Learning
  2. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning
  3. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
  4. 2025Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward
  5. 2025Advancing Multimodal Reasoning via Reinforcement Learning with Cold Start
  6. 2025Mixed-R1: Unified Reward Perspective For Reasoning Capability in Multimodal Large Language Models
  7. 2025Metis-RISE: RL Incentivizes and SFT Enhances Multimodal Reasoning Model Learning
  8. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models