mlsanityData to 14 Oct 2026

Co-Reinforcement Learning for Unified Multimodal Understanding and Generation

arXiv 2505.17534 ↗First posted 23 May 2025cs.CV cs.CL cs.MMcited by 15

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

This paper presents a pioneering exploration of reinforcement learning (RL) via group relative policy optimization for unified multimodal large language models (ULMs), aimed at simultaneously reinforcing generation and understanding capabilities. Through systematic pilot studies, we uncover the significant potential of ULMs to enable the synergistic co-evolution of dual capabilities within a shared policy optimization framework. Building on this insight, we introduce CoRL, a co-reinforcement learning framework comprising a unified RL stage for joint optimization and a refined RL stage for task-specific enhancement. With the proposed CoRL, our resulting model, ULM-R1, achieves average improvements of 7% on three text-to-image generation datasets and 23% on nine multimodal understanding benchmarks. These results demonstrate the effectiveness of CoRL and highlight the substantial benefit of reinforcement learning in facilitating cross-task synergy and optimization for ULMs. Code is available at https://github.com/mm-vl/ULM-R1.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 94 of its 110 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

94 of 110 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models30 Apr
  2. 2025Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning23 Mar
  3. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization17 Mar
  4. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  5. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning10 Mar
  6. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  7. 2024Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning16 May

Other references · 87

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2025Qwen2.5-VL Technical Reportcited by 5,587
  12. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
38 more
  1. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  2. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  3. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
  6. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  7. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  8. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  9. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  10. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  11. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  12. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  13. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  14. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  15. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  16. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  17. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  18. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  19. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  20. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  21. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 586
  22. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 526
  23. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  24. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  25. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  26. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  27. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  28. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 412
  29. 2016Learning to reinforcement learncited by 377
  30. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  31. 2022Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoningcited by 350
  32. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  33. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  34. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  35. 2024We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?cited by 286
  36. 2025VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuningcited by 255
  37. 2024Reinforcement learningcited by 247
  38. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contextscited by 245

50 most cited of 87 All 87 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 7.

15 papers.

Sort:

By area:

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  2. 2025Reconstruction Alignment Improves Unified Multimodal Modelscited by 49
  3. 2025Reinforcement Learning for Large Model: A Surveycited by 9
  4. 2025SUDER: Self-Improving Unified Large Multimodal Models for Understanding and Generation with Dual Self-Rewardscited by 8
  5. 2025MILR: Improving Multimodal Image Generation via Test-Time Latent Reasoningcited by 8
  6. 2025MMaDA-Parallel: Multimodal Large Diffusion Language Models for Thinking-Aware Editing and Generationcited by 8
  7. 2025UniGen-1.5: Enhancing Image Generation and Editing through Reward Unification in Reinforcement Learningcited by 8
  8. 2025PairUni: Pairwise Training for Unified Multimodal Language Modelscited by 1
  9. 2026Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Modelscited by 1
  10. 2026Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinkingcited by 0
  11. 2026From "What" to "How": Constrained Reasoning for Autoregressive Image Generationcited by 0
  12. 2026Towards Customized Multimodal Role-Playcited by 0
3 more
  1. 2026Uni-Synergy: Bridging Understanding and Generation for Personalized Reasoning via Co-operative Reinforcement Learningcited by 0
  2. 2026UniCanvas: A Diffusion-base Unified Model for Text-in-Image Joint Generationcited by 0
  3. 2026From Image Interpretation to Clinical Reasoning: Upstream Physician-Context-Aware Multimodal Learning with Causal Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025UniRL: Self-Improving Unified Multimodal Models via Supervised and Reinforcement Learning
  2. 2026Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimization
  3. 2025OThink-MR1: Stimulating multimodal generalized reasoning capabilities via dynamic reinforcement learning
  4. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods
  5. 2024The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Models
  6. 2026Multi-Task Reinforcement Learning for Enhanced Multimodal LLM-as-a-Judge
  7. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization
  8. 2025SPARK: Synergistic Policy And Reward Co-Evolving Framework