Flow-Factory: A Unified Framework for Reinforcement Learning in Flow-Matching Models
On the map: Diffusion models → Diffusion and flow policies for robotics
also in 1 more
Abstract (arXiv)
Reinforcement learning has emerged as a promising paradigm for aligning diffusion and flow-matching models with human preferences, yet practitioners face fragmented codebases, model-specific implementations, and engineering complexity. We introduce Flow-Factory, a unified framework that decouples algorithms, models, and rewards through through a modular, registry-based architecture. This design enables seamless integration of new algorithms and architectures, as demonstrated by our support for GRPO, DiffusionNFT, and AWM across Flux, Qwen-Image, and WAN video models. By minimizing implementation overhead, Flow-Factory empowers researchers to rapidly prototype and scale future innovations with ease. Flow-Factory provides production-ready memory optimization, flexible multi-reward training, and seamless distributed training support. The codebase is available at https://github.com/X-GenGroup/Flow-Factory.
Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 15 of its 16 references that are in the map. First posted on arXiv 13 Feb 2026.
Builds on
15 of 16 references.
Closest ancestors · 8
Same direction, newest first
- 2025GRPO-Guard: Mitigating Implicit Over-Optimization in Flow Matching via Regulated Clipping25 Oct
- 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Process19 Sep
- 2025Coefficients-Preserving Sampling for Reinforcement Learning with Flow Matching7 Sep
- 2025TempFlow-GRPO: When Timing Matters for GRPO in Flow Models6 Aug
- 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDE29 Jul
- 2025DanceGRPO: Unleashing GRPO on Visual Generation12 May
- 2025Flow-GRPO: Training Flow Matching Models via Online RL8 May
- 2023Training Diffusion Models with Reinforcement Learning22 May
Other references · 7
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
- 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationcited by 193
- 2025Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learningcited by 92
- 2025Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Modelscited by 67
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editingcited by 6
- 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modelingcited by 5
- 2026Scaling Multi-Reference Image Generation with Dynamic Reward Optimizationcited by 3
- 2026RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Datacited by 1
- 2026AdvantageFlow: Advantage-Weighted Least Squares for RL in Flow Modelscited by 1
- 2026CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognitioncited by 0
- 2026NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learningcited by 0
- 2026Manifold Drift in Flow Preference Optimization: A Root Cause of Reward Hackingcited by 0
- 2026λ-Controlled GRPO: Turning Flow-Matching Ratio Instability into a Budgeted Resourcecited by 0
- 2026Decompose Radicals, Then Reward: Fine-Grained Inspection for Accurate Chinese Text Renderingcited by 0
- 2026DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Trainingcited by 0
- 2026Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamicscited by 0
- 2026Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamics9 Oct 2026
- 2026DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training8 Oct 2026
- 2026Decompose Radicals, Then Reward: Fine-Grained Inspection for Accurate Chinese Text Rendering29 Sep 2026
- 2026λ-Controlled GRPO: Turning Flow-Matching Ratio Instability into a Budgeted Resource18 Sep 2026
- 2026Manifold Drift in Flow Preference Optimization: A Root Cause of Reward Hacking20 Aug 2026
- 2026NormGuard: Reward-Preserving Norm Constraints in Flow-Matching Reinforcement Learning26 Jun 2026
- 2026Scaling Multi-Reference Image Generation with Dynamic Reward Optimization25 Jun 2026
- 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editing4 Jun 2026
- 2026AdvantageFlow: Advantage-Weighted Least Squares for RL in Flow Models25 May 2026
- 2026CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognition19 May 2026
- 2026RoboEvolve: Co-Evolving Planner-Simulator for Robotic Manipulation with Limited Data13 May 2026
- 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling30 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2025Flow Matching Policy Gradients
- 2026A Survey of Flow Matching in Reinforcement Learning
- 2026FlowRL: A Taxonomy and Modular Framework for Reinforcement Learning with Diffusion Policies
- 2025Reinforcement Learning for Flow-Matching Policies
- 2026Flow Matching for Offline Reinforcement Learning with Discrete Actions
- 2026Faster and Better Alignment for Flow Matching Models via Step-aware Advantages
- 2026FlowCPO: A Unified Divergence View of Preference Alignment for Flow Models
- 2026MEND: RL For Flow Models via Proximal Velocity Matching