mlsanityData to 14 Oct 2026

Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation

arXiv 2510.13418 ↗First posted 15 Oct 2025cs.CVcited by 14

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning (RL) has garnered increasing attention in text-to-image (T2I) generation. However, most existing RL approaches are tailored to either diffusion models or autoregressive models, overlooking an important alternative: masked generative models. In this work, we propose Mask-GRPO, the first method to incorporate Group Relative Policy Optimization (GRPO)-based RL into this overlooked paradigm. Our core insight is to redefine the transition probability, which is different from current approaches, and formulate the unmasking process as a multi-step decision-making problem. To further enhance our method, we explore several useful strategies, including removing the KL constraint, applying the reduction strategy, and filtering out low-quality samples. Using Mask-GRPO, we improve a base model, Show-o, with substantial improvements on standard T2I benchmarks and preference alignment, outperforming existing state-of-the-art approaches. The code is available on https://github.com/xingzhejun/Mask-GRPO

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 51 of its 87 references that are in the map. First posted on arXiv 15 Oct 2025.

Builds on

51 of 87 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025DanceGRPO: Unleashing GRPO on Visual Generation12 May
  2. 2025Flow-GRPO: Training Flow Matching Models via Online RL8 May
  3. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  6. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 45

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022Flow Matching for Generative Modelingcited by 5,702
  7. 2024GPT-4o System Cardcited by 4,588
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  9. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  10. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  11. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  12. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
33 more
  1. 2024OpenAI o1 System Cardcited by 1,244
  2. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  3. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  4. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
  5. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  6. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  7. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  8. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  9. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  10. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  11. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 370
  12. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  13. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 323
  14. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
  15. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  16. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 289
  17. 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 163
  18. 2023Emu: Generative Pretraining in Multimodalitycited by 157
  19. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 131
  20. 2025OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cyclescited by 124
  21. 2024mDPO: Conditional Preference Optimization for Multimodal Large Language Modelscited by 92
  22. 2025SimpleAR: Pushing the Frontier of Autoregressive Visual Generation through Pretraining, SFT, and RLcited by 86
  23. 2025GPT-ImgEval: A Comprehensive Benchmark for Diagnosing GPT4o in Image Generationcited by 58
  24. 2024Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesiscited by 56
  25. 2025Democratizing Text-to-Image Masked Generative Models with Compact Text-Aware One-Dimensional Tokenscited by 50
  26. 2022Unified Discrete Diffusion for Simultaneous Vision-Language Generationcited by 39
  27. 2025Safety Reasoning with Guidelinescited by 15
  28. 2025Lifelong Safety Alignment for Language Modelscited by 15
  29. 2024Step-On-Feet Tuning: Scaling Self-Alignment of LLMs via Bootstrappingcited by 14
  30. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 14
  31. 2024Probing the Safety Response Boundary of Large Language Models via Unsafe Decoding Path Generationcited by 10
  32. 2025PerPO: Perceptual Preference Optimization via Discriminative Rewardingcited by 9
  33. 2025Wavelet Fourier Diffuser: Frequency-Aware Diffusion Model for Reinforcement Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 12.

14 papers.

Sort:

By area:

  1. 2025Principled RL for Flow Matching Emerges from the Chunk-level Policy Optimizationcited by 14
  2. 2026OmniNFT: Modality-wise Omni Diffusion Reinforcement for Joint Audio-Video Generationcited by 7
  3. 2025Co-GRPO: Co-Optimized Group Relative Policy Optimization for Masked Diffusion Modelcited by 6
  4. 2026UDM-GRPO: Stable and Efficient Group Relative Policy Optimization for Uniform Discrete Diffusion Modelscited by 3
  5. 2026Power Reinforcement Post-Training of Text-to-Image Models with Super-Linear Advantage Shapingcited by 3
  6. 2026Advances in GRPO for Generation Models: A Surveycited by 2
  7. 2026MAR-GRPO: Stabilized GRPO for AR-diffusion Hybrid Image Generationcited by 2
  8. 2026dFlowGRPO: Rate-Aware Policy Optimization for Discrete Flow Modelscited by 2
  9. 2026Language-based Trial and Error Falls Behind in the Era of Experiencecited by 1
  10. 2026JAGG: Jacobian-Aggregated Group Gradient for Efficient GRPO Training of Diffusion Modelscited by 0
  11. 2026Discrete Diffusion Bridges for Spatiotemporally Aligned Image Translation and Generationcited by 0
  12. 2026Balancing Emotional Alignment and Semantic Consistency in Image Generation via Reinforcement Learning with Valence-Arousal Anchoringcited by 0
2 more
  1. 2026MaskSense: Confronting the Visual Exploration Trap in Masked Image Generationcited by 0
  2. 2026Size Generalization for Diffusion Routing Solverscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025AR-GRPO: Training Autoregressive Image Generation Models via Reinforcement Learning
  2. 2025MaskFocus: Focusing Policy Optimization on Critical Steps for Masked Image Generation
  3. 2025Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPO
  4. 2025Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning
  5. 2026V-GRPO: Online Reinforcement Learning for Denoising Generative Models Is Easier than You Think
  6. 2025STAGE: Stable and Generalizable GRPO for Autoregressive Image Generation
  7. 2026GRPO-TTA: Test-Time Visual Tuning for Vision-Language Models via GRPO-Driven Reinforcement Learning
  8. 2025GRPO-RM: Fine-Tuning Representation Models via GRPO-Driven Reinforcement Learning