mlsanityData to 14 Oct 2026

Principled RL for Flow Matching Emerges from the Chunk-level Policy Optimization

arXiv 2510.21583 ↗First posted 24 Oct 2025cs.CV cs.AIcited by 14

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Recent Progress in post-training flow matching for text-to-image (T2I) generation with Group Relative Policy Optimization (GRPO) has demonstrated strong potential. However, it is hindered by a critical limitation: inaccurate advantage attribution. In this work, we argue that aggregating consecutive steps into a coherent 'chunk' and shifting the policy optimization paradigm from GRPO's step level to the chunk level can effectively mitigate the negative impact of this issue. Building on this insight, we propose Group Chunking Policy Optimization (GCPO), the first chunk-level reinforcement learning approach for post-training flow matching. Extensive experiments demonstrate that GCPO achieves superior performance on both standard T2I benchmarks and preference alignment, with up to 43% relative gains over GRPO, highlighting the promise of chunk-level policy optimization. The code is available on https://github.com/xingzhejun/GCPO.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 42 of its 52 references that are in the map. First posted on arXiv 24 Oct 2025.

Builds on

42 of 52 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation15 Oct
  2. 2025Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning28 Aug
  3. 2025TempFlow-GRPO: When Timing Matters for GRPO in Flow Models6 Aug
  4. 2025DanceGRPO: Unleashing GRPO on Visual Generation12 May
  5. 2025Flow-GRPO: Training Flow Matching Models via Online RL8 May
  6. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoning13 Jan
  7. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 35

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  8. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  9. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  10. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  11. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  12. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
23 more
  1. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  2. 2025Qwen-Image Technical Reportcited by 1,044
  3. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  4. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  5. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  6. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  7. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  8. 2025Group Sequence Policy Optimizationcited by 835
  9. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
  10. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
  11. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  12. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
  13. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 269
  14. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 211
  15. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 178
  16. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Cachingcited by 151
  17. 2025Reinforcement Learning with Action Chunkingcited by 107
  18. 2024Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimizationcited by 84
  19. 2025BranchGRPO: Stable and Efficient GRPO with Structured Branching in Diffusion Modelscited by 61
  20. 2025Coefficients-Preserving Sampling for Reinforcement Learning with Flow Matchingcited by 56
  21. 2024Generalizing Alignment Paradigm of Text-to-Image Generation with Preferences through f-divergence Minimizationcited by 15
  22. 2025Lifelong Safety Alignment for Language Modelscited by 15
  23. 2025Reinforcement Fine-Tuning Powers Reasoning Capability of Multimodal Large Language Modelscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 2026E-GRPO: High Entropy Steps Drive Effective Reinforcement Learning for Flow Modelscited by 14
  2. 2025VQRAE: Representation Quantization Autoencoders for Multimodal Understanding, Generation and Reconstructioncited by 10
  3. 2026RAVEN: Real-time Autoregressive Video Extrapolation with Consistency-model GRPOcited by 7
  4. 2026HP-Edit: A Human-Preference Post-Training Framework for Image Editingcited by 6
  5. 2026Diffusion Reinforcement Learning via Centered Reward Distillationcited by 5
  6. 2025SoliReward: Mitigating Susceptibility to Reward Hacking and Annotation Noise in Video Generation Reward Modelscited by 4
  7. 2026Power Reinforcement Post-Training of Text-to-Image Models with Super-Linear Advantage Shapingcited by 3
  8. 2026Euphonium: Steering Video Flow Matching via Process Reward Gradient Guided Stochastic Dynamicscited by 2
  9. 2026Advances in GRPO for Generation Models: A Surveycited by 2
  10. 2025RePack then Refine: Efficient Diffusion Transformer with Vision Foundation Modelcited by 1
  11. 2026Balancing Performance and Diversity in GRPO Autoregressive Text-to-Image Post-Trainingcited by 0
  12. 2026LeanGRPO: Eliminating Redundant Recomputation in Diffusion RLcited by 0
2 more
  1. 2026Flow Matching Reinforcement for 3D Mesh Generation via Dynamic Homing Optimizationcited by 0
  2. 2026FlowCred: Hierarchical Credit Assignment and Functional Coordination for Joint Text-to-Video Post-Trainingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Faster and Better Alignment for Flow Matching Models via Step-aware Advantages
  2. 2025GRPO-Guard: Mitigating Implicit Over-Optimization in Flow Matching via Regulated Clipping
  3. 2025Smart-GRPO: Smartly Sampling Noise for Efficient RL of Flow-Matching Models
  4. 2026OP-GRPO: Efficient Off-Policy GRPO for Flow-Matching Models
  5. 2026From Sparse to Dense: Multi-View GRPO for Flow Models via Augmented Condition Space
  6. 2025Anchoring Values in Temporal and Group Dimensions for Flow Matching Model Alignment
  7. 2025SuperFlow: Training Flow Matching Models with RL on the Fly
  8. 2026Unified Trajectory Matching Policy Optimization: Diverse T2I Generation and VLA Generalization