# ReasonGen-R1: CoT for Autoregressive Image generation models through SFT and RL

arXiv: https://arxiv.org/abs/2505.24875 · HTML: https://mlsanity.com/paper/2505.24875/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 18 of its 49 references that are in the map.
- First posted on arXiv 30 May 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 18 of 18)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Show-o: One Single Transformer to Unify Multimodal Understanding and Generation](https://mlsanity.com/paper/2408.12528/) (2024)
- [R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization](https://mlsanity.com/paper/2503.10615/) (2025)
- [Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models](https://mlsanity.com/paper/2403.18814/) (2024)
- [MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-based Reinforcement Learning](https://mlsanity.com/paper/2503.07365/) (2025)
- [R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization](https://mlsanity.com/paper/2503.12937/) (2025)
- [DreamLLM: Synergistic Multimodal Comprehension and Creation](https://mlsanity.com/paper/2309.11499/) (2023)
- [MetaMorph: Multimodal Understanding and Generation via Instruction Tuning](https://mlsanity.com/paper/2412.14164/) (2024)
- [Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step](https://mlsanity.com/paper/2501.13926/) (2025)
- [OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles](https://mlsanity.com/paper/2503.17352/) (2025)
- [Dual Diffusion for Unified Image Generation and Understanding](https://mlsanity.com/paper/2501.00289/) (2024)
- [ILLUME: Illuminating Your LLMs to See, Draw, and Self-Enhance](https://mlsanity.com/paper/2412.06673/) (2024)

## Built on by (most cited first, 30 of 30)

- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [Visual Autoregressive Modeling for Instruction-Guided Image Editing](https://mlsanity.com/paper/2508.15772/) (2025)
- [Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data](https://mlsanity.com/paper/2511.12609/) (2025)
- [Dynamic-TreeRPO: Breaking the Independent Trajectory Bottleneck with Structured Sampling](https://mlsanity.com/paper/2509.23352/) (2025)
- [Group Critical-token Policy Optimization for Autoregressive Image Generation](https://mlsanity.com/paper/2509.22485/) (2025)
- [DraCo: Draft as CoT for Text-to-Image Preview and Rare Concept Generation](https://mlsanity.com/paper/2512.05112/) (2025)
- [Integrating Reinforcement Learning with Visual Generative Models: Foundations and Advances](https://mlsanity.com/paper/2508.10316/) (2025)
- [Understanding-in-Generation: Reinforcing Generative Capability of Unified Model via Infusing Understanding into Generation](https://mlsanity.com/paper/2509.18639/) (2025)
- [Visual-Aware CoT: Achieving High-Fidelity Visual Consistency in Unified Models](https://mlsanity.com/paper/2512.19686/) (2025)
- [Reinforcement Learning for Large Model: A Survey](https://mlsanity.com/paper/2508.08189/) (2025)
- [MILR: Improving Multimodal Image Generation via Test-Time Latent Reasoning](https://mlsanity.com/paper/2509.22761/) (2025)
- [Improving Chain-of-Thought Efficiency for Autoregressive Image Generation](https://mlsanity.com/paper/2510.05593/) (2025)
- [From Broad Exploration to Stable Synthesis: Entropy-Guided Optimization for Autoregressive Image Generation](https://mlsanity.com/paper/2604.02355/) (2026)
- [Think in Strokes, Not Pixels: Process-Driven Image Generation via Interleaved Reasoning](https://mlsanity.com/paper/2604.04746/) (2026)
- [Beyond Textual CoT: Interleaved Text-Image Chains with Deep Confidence Reasoning for Image Editing](https://mlsanity.com/paper/2510.08157/) (2025)
- [Expand and Prune: Maximizing Trajectory Diversity for Effective GRPO in Generative Models](https://mlsanity.com/paper/2512.15347/) (2025)
- [Beyond Unimodal Shortcuts: MLLMs as Cross-Modal Reasoners for Grounded Named Entity Recognition](https://mlsanity.com/paper/2602.04486/) (2026)
- [CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation](https://mlsanity.com/paper/2603.08652/) (2026)
- [Understanding vs. Generation: Navigating Optimization Dilemma in Multimodal Models](https://mlsanity.com/paper/2602.15772/) (2026)
- [Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling](https://mlsanity.com/paper/2604.28185/) (2026)
- [MiLDEdit: Reasoning-Based Multi-Layer Design Document Editing](https://mlsanity.com/paper/2601.04589/) (2026)
- [StruVis: Enhancing Reasoning-based Text-to-Image Generation via Thinking with Structured Vision](https://mlsanity.com/paper/2603.06032/) (2026)
- [FiRe: Fine-grained Multimodal Reasoning for Enhanced Image Generation](https://mlsanity.com/paper/2604.13491/) (2026)
- [Probing Visual Planning in Image Editing Models](https://mlsanity.com/paper/2604.22868/) (2026)
- [Compositional Text-to-Image Generation Via Region-aware Bimodal Direct Preference Optimization](https://mlsanity.com/paper/2605.28615/) (2026)
- [IV-CoT: Implicit Visual Chain-of-Thought for Structure-Aware Text-to-Image Generation](https://mlsanity.com/paper/2606.24849/) (2026)
- [GenPuzzle: Benchmarking Visual Reasoning in Image Generation Models](https://mlsanity.com/paper/2609.05902/) (2026)
- [Agentic Visual Generation: From Generative Models to Agentic Control](https://mlsanity.com/paper/2609.06758/) (2026)
- [Uni-Synergy: Bridging Understanding and Generation for Personalized Reasoning via Co-operative Reinforcement Learning](https://mlsanity.com/paper/2605.10445/) (2026)
- [Learning Native Reflection in Unified Models with Interleaved Reinforcement Learning](https://mlsanity.com/paper/2609.35767/) (2026)

## Abstract (arXiv)

Although chain-of-thought reasoning and reinforcement learning (RL) have driven breakthroughs in NLP, their integration into generative vision models remains underexplored. We introduce ReasonGen-R1, a two-stage framework that first imbues an autoregressive image generator with explicit text-based "thinking" skills via supervised fine-tuning on a newly generated reasoning dataset of written rationales, and then refines its outputs using Group Relative Policy Optimization. To enable the model to reason through text before generating images, We automatically generate and release a corpus of model crafted rationales paired with visual prompts, enabling controlled planning of object layouts, styles, and scene compositions. Our GRPO algorithm uses reward signals from a pretrained vision language model to assess overall visual quality, optimizing the policy in each update. Evaluations on GenEval, DPG, and the T2I benchmark demonstrate that ReasonGen-R1 consistently outperforms strong baselines and prior state-of-the-art models. More: aka.ms/reasongen.