Janus-Pro-R1: Advancing Collaborative Visual Comprehension and Generation via Reinforcement Learning
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
also in 1 more
Abstract (arXiv)
Recent endeavors in Multimodal Large Language Models (MLLMs) aim to unify visual comprehension and generation. However, these two capabilities remain largely independent, as if they are two separate functions encapsulated within the same model. Consequently, visual comprehension does not enhance visual generation, and the reasoning mechanisms of LLMs have not been fully integrated to revolutionize image generation. In this paper, we propose to enable the collaborative co-evolution of visual comprehension and generation, advancing image generation into an iterative introspective process. We introduce a two-stage training approach: supervised fine-tuning teaches the MLLM with the foundational ability to generate genuine CoT for visual generation, while reinforcement learning activates its full potential via an exploration-exploitation trade-off. Ultimately, we unlock the Aha moment in visual generation, advancing MLLMs from text-to-image tasks to unified image generation. Extensive experiments demonstrate that our model not only excels in text-to-image generation and image editing, but also functions as a superior image semantic evaluator with enhanced visual comprehension capabilities. Project Page: https://janus-pro-r1.github.io.
Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 51 of its 61 references that are in the map. First posted on arXiv 2 Jun 2025.
Builds on
51 of 61 references.
Closest ancestors · 2
Same direction, newest first
- 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editing13 Mar
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation28 Jan
Other references · 49
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Visual Instruction Tuningcited by 7,869
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2023Qwen Technical Reportcited by 3,344
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
37 more
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
- 2023Generative Multimodal Models are In-Context Learnerscited by 446
- 2024OmniGen: Unified Image Generationcited by 394
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
- 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
- 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
- 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 241
- 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
- 2024World Model on Million-Length Video And Language With Blockwise RingAttentioncited by 198
- 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 171
- 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 166
- 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
- 2023InstructDiffusion: A Generalist Modeling Interface for Vision Taskscited by 164
- 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 131
- 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 77
- 2025EditAR: Unified Conditional Generation with Autoregressive Modelscited by 40
- 2025Selftok: Discrete Visual Tokens of Autoregression, by Diffusion, and for Reasoningcited by 33
- 2024Auto-Encoding Morph-Tokens for Multimodal LLMcited by 30
- 2025Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokenscited by 28
- 2024STEP: Enhancing Video-LLMs' Compositional Reasoning by Spatio-Temporal Graph-guided Self-Trainingcited by 22
- 2021Exploration-exploitation trade-off for continuous-time episodic reinforcement learning with linear-convex modelscited by 21
- 2025VARGPT-v1.1: Improve Visual Autoregressive Large Unified Model via Iterative Instruction Tuning and Reinforcement Learningcited by 21
- 2022Introspective Learning : A Two-Stage Approach for Inference in Neural Networkscited by 20
- 2024Multi-Reward as Condition for Instruction-based Image Editingcited by 20
- 2024Towards Unified Multimodal Editing with Enhanced Knowledge Collaborationcited by 18
- 2025Reasoning Physical Video Generation with Diffusion Timestep Tokens via Reinforcement Learningcited by 18
- 2023Self-supervised Meta-Prompt Learning with Meta-Gradient Regularization for Few-shot Generalizationcited by 15
- 2024Unified Generative and Discriminative Training for Multi-modal Large Language Modelscited by 15
Built on by
Citing papers per year: 2025: 5; 2026: 4.
9 papers.
- 2025CodePlot-CoT: Mathematical Visual Reasoning by Thinking with Code-Driven Imagescited by 23
- 2025FocusDiff: Advancing Fine-Grained Text-Image Alignment for Autoregressive Visual Generation through RLcited by 13
- 2025WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editingcited by 11
- 2025What Limits Virtual Agent Application? OmniBench: A Scalable Multi-Dimensional Benchmark for Essential Virtual Agent Capabilitiescited by 8
- 2026VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generationcited by 5
- 2026MapDream: Task-Driven Map Learning for Vision-Language Navigationcited by 4
- 2025OmniMoGen: Unifying Human Motion Generation via Learning from Interleaved Text-Motion Instructionscited by 1
- 2026Guidance Contrastive Token Credit Assignment for Discrete Policy Optimizationcited by 0
- 2026Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Trainingcited by 0
- 2026Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training4 Oct 2026
- 2026VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation19 Aug 2026
- 2026Guidance Contrastive Token Credit Assignment for Discrete Policy Optimization28 May 2026
- 2026MapDream: Task-Driven Map Learning for Vision-Language Navigation30 Jan 2026
- 2025OmniMoGen: Unifying Human Motion Generation via Learning from Interleaved Text-Motion Instructions22 Dec 2025
- 2025WiseEdit: Benchmarking Cognition- and Creativity-Informed Image Editing29 Nov 2025
- 2025CodePlot-CoT: Mathematical Visual Reasoning by Thinking with Code-Driven Images13 Oct 2025
- 2025What Limits Virtual Agent Application? OmniBench: A Scalable Multi-Dimensional Benchmark for Essential Virtual Agent Capabilities10 Jun 2025
- 2025FocusDiff: Advancing Fine-Grained Text-Image Alignment for Autoregressive Visual Generation through RL5 Jun 2025
Similar, not cited
Close in wording, no citation either way
- 2025GoT-R1: Unleashing Reasoning Capability of MLLM for Visual Generation with Reinforcement Learning
- 2025ThinkGen: Generalized Thinking for Visual Generation
- 2026Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models
- 2025Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?
- 2026FiRe: Fine-grained Multimodal Reasoning for Enhanced Image Generation
- 2025Learning Only with Images: Visual Reinforcement Learning with Reasoning, Rendering, and Visual Feedback
- 2025Visual-CoG: Stage-Aware Reinforcement Learning with Chain of Guidance for Text-to-Image Generation