mlsanityData to 14 Oct 2026

CFlowNets: Continuous Control with Generative Flow Networks

arXiv 2303.02430 ↗First posted 4 Mar 2023cs.LGcited by 19

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Generative flow networks (GFlowNets), as an emerging technique, can be used as an alternative to reinforcement learning for exploratory control tasks. GFlowNet aims to generate distribution proportional to the rewards over terminating states, and to sample different candidates in an active learning fashion. GFlowNets need to form a DAG and compute the flow matching loss by traversing the inflows and outflows of each node in the trajectory. No experiments have yet concluded that GFlowNets can be used to handle continuous tasks. In this paper, we propose generative continuous flow networks (CFlowNets) that can be applied to continuous control tasks. First, we present the theoretical formulation of CFlowNets. Then, a training framework for CFlowNets is proposed, including the action selection process, the flow approximation algorithm, and the continuous flow matching loss function. Afterward, we theoretically prove the error bound of the flow approximation. The error decreases rapidly as the number of flow samples increases. Finally, experimental results on continuous control tasks demonstrate the performance advantages of CFlowNets compared to many reinforcement learning methods, especially regarding exploration ability.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 28 of its 41 references that are in the map. First posted on arXiv 4 Mar 2023.

Builds on

28 of 41 references.

References · 28

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  5. 2015Trust Region Policy Optimizationcited by 2,580
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  7. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  8. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 787
  9. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  10. 2018Learning Dexterous In-Hand Manipulationcited by 687
  11. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 574
  12. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 392
16 more
  1. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 311
  2. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 251
  3. 2018Provably Efficient Maximum Entropy Explorationcited by 242
  4. 2018The Convergence of Sparsified Gradient Methodscited by 229
  5. 2021Reinforcement Learning with Prototypical Representationscited by 179
  6. 2022Biological Sequence Design with GFlowNetscited by 174
  7. 2022Bayesian Structure Learning with Generative Flow Networkscited by 153
  8. 2020Phasic Policy Gradientcited by 123
  9. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 114
  10. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 102
  11. 2018Evolution-Guided Policy Gradient in Reinforcement Learningcited by 83
  12. 2017Virtual to Real Reinforcement Learning for Autonomous Drivingcited by 80
  13. 2018Latent Space Policies for Hierarchical Reinforcement Learningcited by 67
  14. 2018Accelerated Methods for Deep Reinforcement Learningcited by 40
  15. 2022Unifying Generative Models with GFlowNets and Beyondcited by 24
  16. 2022GFlowCausal: Generative Flow Networks for Causal Discoverycited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 5; 2025: 4; 2026: 1.

19 papers.

Sort:

By area:

  1. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 112
  2. 2023Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimizationcited by 57
  3. 2023Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Networkcited by 50
  4. 2024Discrete Probabilistic Inference as Control in Multi-path Environmentscited by 44
  5. 2024Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examplescited by 30
  6. 2025Generative Models in Decision Making: A Surveycited by 15
  7. 2023Generative Flow Networks: a Markov Chain Perspectivecited by 14
  8. 2023A Theory of Non-Acyclic Generative Flow Networkscited by 12
  9. 2024A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challengescited by 10
  10. 2023Generalized Universal Domain Adaptation with Generative Flow Networkscited by 7
  11. 2025Hybrid-Balance GFlowNet for Solving Vehicle Routing Problemscited by 7
  12. 2023Generative Flow Networks for Precise Reward-Oriented Active Learning on Graphscited by 3
7 more
  1. 2023GFlowNets with Human Feedbackcited by 3
  2. 2024Multi-Agent Continuous Control with Generative Flow Networkscited by 3
  3. 2024Efficient Symmetry-Aware Materials Generation via Hierarchical Generative Flow Networkscited by 3
  4. 2025Ergodic Generative Flowscited by 2
  5. 2023Meta Generative Flow Networks with Personalization for Task-Specific Adaptationcited by 1
  6. 2025Improved Exploration in GFlownets via Enhanced Epistemic Neural Networkscited by 1
  7. 2026WINFlowNets: Warm-up Integrated Networks Training of Generative Flow Networks for Robotics and Machine Fault Adaptationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024GFlowNet Training by Policy Gradients
  2. 2024Improving GFlowNets with Monte Carlo Tree Search
  3. 2024Optimizing Backward Policies in GFlowNets via Trajectory Likelihood Maximization
  4. 2025A Theory of Multi-Agent Generative Flow Networks
  5. 2025Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Training
  6. 2024Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNets
  7. 2026Towards Understanding the Training of GFlowNets: A Theoretical Perspective
  8. 2026Avoid What You Know: Divergent Trajectory Balance for GFlowNets