mlsanityData to 7 Oct 2026

GFlowNet Training by Policy Gradients

arXiv 2408.05885 ↗First posted 12 Aug 2024cs.LG stat.MLcited by 4

Not on the map. Why?

Abstract (arXiv)

Generative Flow Networks (GFlowNets) have been shown effective to generate combinatorial objects with desired properties. We here propose a new GFlowNet training framework, with policy-dependent rewards, that bridges keeping flow balance of GFlowNets to optimizing the expected accumulated reward in traditional Reinforcement-Learning (RL). This enables the derivation of new policy-based GFlowNet training methods, in contrast to existing ones resembling value-based RL. It is known that the design of backward policies in GFlowNet training affects efficiency. We further develop a coupled training strategy that jointly solves GFlowNet forward policy training and backward policy design. Performance analysis is provided with a theoretical guarantee of our policy-based GFlowNet training. Experiments on both simulated and real-world datasets verify that our policy-based strategies provide advanced RL perspectives for robust gradient estimation to improve GFlowNet performance.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 33 of its 47 references that are in the map. First posted on arXiv 12 Aug 2024.

Builds on

33 of 47 references.

References · 33

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2015Trust Region Policy Optimizationcited by 2,503
  6. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  7. 2016Generative Adversarial Imitation Learningcited by 1,576
  8. 2017Constrained Policy Optimizationcited by 928
  9. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  10. 2015Importance Weighted Autoencoderscited by 454
  11. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  12. 2021GFlowNet Foundationscited by 286
21 more
  1. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  2. 2017Advances in Variational Inferencecited by 236
  3. 2018Approximation Methods for Bilevel Programmingcited by 231
  4. 2020Bilevel Optimization: Convergence Analysis and Enhanced Designcited by 219
  5. 2017Equivalence Between Policy Gradients and Soft Q-Learningcited by 172
  6. 2022Bayesian Structure Learning with Generative Flow Networkscited by 153
  7. 2020A Two-Timescale Framework for Bilevel Optimization: Complexity Analysis and Application to Actor-Criticcited by 141
  8. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 129
  9. 2022GFlowNets and variational inferencecited by 103
  10. 2020Toward the Fundamental Limits of Imitation Learningcited by 96
  11. 2021Policy Mirror Descent for Regularized Reinforcement Learning: A Generalized Framework with Linear Convergencecited by 83
  12. 2023Better Training of GFlowNets with Local Credit and Incomplete Trajectoriescited by 83
  13. 2023Towards Understanding and Improving GFlowNet Trainingcited by 74
  14. 2022Reinforcement Learning with Sparse Rewards using Guidance from Offline Demonstrationcited by 54
  15. 2023Local Search GFlowNetscited by 45
  16. 2022A Variational Perspective on Generative Flow Networkscited by 39
  17. 2014Addendum on the scoring of Gaussian directed acyclic graphical modelscited by 32
  18. 2023Thompson sampling for improved exploration in GFlowNetscited by 31
  19. 2023Learning to Scale Logits for Temperature-Conditional GFlowNetscited by 26
  20. 2023Learning Energy Decompositions for Partial Inference of GFlowNetscited by 24
  21. 2021Nested Variational Inferencecited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Rooted Absorbed Prefix Trajectory Balance with Submodular Replay for GFlowNet Trainingcited by 2
  2. 2026Evaluating GFlowNet from partial episodes for stable and flexible policy-based trainingcited by 1
  3. 2026WINFlowNets: Warm-up Integrated Networks Training of Generative Flow Networks for Robotics and Machine Fault Adaptationcited by 0
  4. 2026Proximal Policy Optimization for Amortized Discrete Samplingcited by 0