# Flow-Factory: A Unified Framework for Reinforcement Learning in Flow-Matching Models

arXiv: https://arxiv.org/abs/2602.12529 · HTML: https://mlsanity.com/paper/2602.12529/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models.
- Builds on 15 of its 16 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Placed in

- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)
- Deep reinforcement learning methods / [RL with diffusion and flow models](https://mlsanity.com/deeprl/rl-with-diffusion-and-flow-models/)

## Builds on (most cited first, 15 of 15)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Training Diffusion Models with Reinforcement Learning](https://mlsanity.com/paper/2305.13301/) (2023)
- [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2305.01569/) (2023)
- [Flow-GRPO: Training Flow Matching Models via Online RL](https://mlsanity.com/paper/2505.05470/) (2025)
- [DanceGRPO: Unleashing GRPO on Visual Generation](https://mlsanity.com/paper/2505.07818/) (2025)
- [DiffusionNFT: Online Diffusion Reinforcement with Forward Process](https://mlsanity.com/paper/2509.16117/) (2025)
- [GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization](https://mlsanity.com/paper/2601.05242/) (2026)
- [MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE](https://mlsanity.com/paper/2507.21802/) (2025)
- [TempFlow-GRPO: When Timing Matters for GRPO in Flow Models](https://mlsanity.com/paper/2508.04324/) (2025)
- [Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning](https://mlsanity.com/paper/2508.20751/) (2025)
- [Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Models](https://mlsanity.com/paper/2509.25050/) (2025)
- [Coefficients-Preserving Sampling for Reinforcement Learning with Flow Matching](https://mlsanity.com/paper/2509.05952/) (2025)
- [GRPO-Guard: Mitigating Implicit Over-Optimization in Flow Matching via Regulated Clipping](https://mlsanity.com/paper/2510.22319/) (2025)

## Built on by (most cited first, 13 of 13)

- [LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing](https://mlsanity.com/paper/2606.06042/) (2026)
- [Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling](https://mlsanity.com/paper/2604.28185/) (2026)
- [Scaling Multi-Reference Image Generation with Dynamic Reward Optimization](https://mlsanity.com/paper/2606.26947/) (2026)
- [RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Data](https://mlsanity.com/paper/2605.13775/) (2026)
- [AdvantageFlow: Advantage-Weighted Least Squares for RL in Flow Models](https://mlsanity.com/paper/2605.26013/) (2026)
- [CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognition](https://mlsanity.com/paper/2605.19995/) (2026)
- [NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learning](https://mlsanity.com/paper/2606.27771/) (2026)
- [Manifold Drift in Flow Preference Optimization: A Root Cause of Reward Hacking](https://mlsanity.com/paper/2608.20011/) (2026)
- [λ-Controlled GRPO: Turning Flow-Matching Ratio Instability into a Budgeted Resource](https://mlsanity.com/paper/2609.22041/) (2026)
- [Decompose Radicals, Then Reward: Fine-Grained Inspection for Accurate Chinese Text Rendering](https://mlsanity.com/paper/2609.37569/) (2026)
- [DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training](https://mlsanity.com/paper/2610.12468/) (2026)
- [Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamics](https://mlsanity.com/paper/2610.13522/) (2026)
- [Blame Errant Words, Not the Canvas: Defect-Aware Spatial Credit Assignment for Visual Text Generation](https://mlsanity.com/paper/2610.20794/) (2026)

## Abstract (arXiv)

Reinforcement learning has emerged as a promising paradigm for aligning diffusion and flow-matching models with human preferences, yet practitioners face fragmented codebases, model-specific implementations, and engineering complexity. We introduce Flow-Factory, a unified framework that decouples algorithms, models, and rewards through through a modular, registry-based architecture. This design enables seamless integration of new algorithms and architectures, as demonstrated by our support for GRPO, DiffusionNFT, and AWM across Flux, Qwen-Image, and WAN video models. By minimizing implementation overhead, Flow-Factory empowers researchers to rapidly prototype and scale future innovations with ease. Flow-Factory provides production-ready memory optimization, flexible multi-reward training, and seamless distributed training support. The codebase is available at https://github.com/X-GenGroup/Flow-Factory.