mlsanityData to 14 Oct 2026

Janus-Pro-R1: Advancing Collaborative Visual Comprehension and Generation via Reinforcement Learning

arXiv 2506.01480 ↗First posted 2 Jun 2025cs.CVcited by 9

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

also in 1 more
Abstract (arXiv)

Recent endeavors in Multimodal Large Language Models (MLLMs) aim to unify visual comprehension and generation. However, these two capabilities remain largely independent, as if they are two separate functions encapsulated within the same model. Consequently, visual comprehension does not enhance visual generation, and the reasoning mechanisms of LLMs have not been fully integrated to revolutionize image generation. In this paper, we propose to enable the collaborative co-evolution of visual comprehension and generation, advancing image generation into an iterative introspective process. We introduce a two-stage training approach: supervised fine-tuning teaches the MLLM with the foundational ability to generate genuine CoT for visual generation, while reinforcement learning activates its full potential via an exploration-exploitation trade-off. Ultimately, we unlock the Aha moment in visual generation, advancing MLLMs from text-to-image tasks to unified image generation. Extensive experiments demonstrate that our model not only excels in text-to-image generation and image editing, but also functions as a superior image semantic evaluator with enhanced visual comprehension capabilities. Project Page: https://janus-pro-r1.github.io.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 51 of its 61 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

51 of 61 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing13 Mar
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan

Other references · 49

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  8. 2023Qwen Technical Reportcited by 3,344
  9. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  10. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  12. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
37 more
  1. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  2. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  3. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  4. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  5. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  6. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  7. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  8. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  9. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  10. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  11. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  12. 2024OmniGen: Unified Image Generationcited by 394
  13. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  14. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
  15. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
  16. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 241
  17. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  18. 2024World Model on Million-Length Video And Language With Blockwise RingAttentioncited by 198
  19. 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 171
  20. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 166
  21. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  22. 2023InstructDiffusion: A Generalist Modeling Interface for Vision Taskscited by 164
  23. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 131
  24. 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 77
  25. 2025EditAR: Unified Conditional Generation with Autoregressive Modelscited by 40
  26. 2025Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoningcited by 33
  27. 2024Auto-Encoding Morph-Tokens for Multimodal LLMcited by 30
  28. 2025Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokenscited by 28
  29. 2024STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Trainingcited by 22
  30. 2021Exploration-exploitation trade-off for continuous-time episodic reinforcement learning with linear-convex modelscited by 21
  31. 2025VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learningcited by 21
  32. 2022Introspective Learning : A Two-Stage Approach for Inference in Neural Networkscited by 20
  33. 2024Multi-Reward as Condition for Instruction-based Image Editingcited by 20
  34. 2024Towards Unified Multimodal Editing with Enhanced Knowledge Collaborationcited by 18
  35. 2025Reasoning Physical Video Generation with Diffusion Timestep Tokens via Reinforcement Learningcited by 18
  36. 2023Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalizationcited by 15
  37. 2024Unified Generative and Discriminative Training for Multi-modal Large Language Modelscited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 4.

9 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation with Reinforcement Learning
  2. 2025ThinkGen: Generalized Thinking for Visual Generation
  3. 2026Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models
  4. 2025Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation
  5. 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?
  6. 2026FiRe: Fine-grained Multimodal Reasoning for Enhanced Image Generation
  7. 2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback
  8. 2025Visual-CoG: Stage-Aware Reinforcement Learning with Chain of Guidance for Text-to-Image Generation