mlsanityData to 14 Oct 2026

Learning GFlowNets from partial episodes for improved convergence and stability

arXiv 2209.12782 ↗First posted 26 Sep 2022cs.LG stat.MLcited by 132

Not on the map. Why?

Abstract (arXiv)

Generative flow networks (GFlowNets) are a family of algorithms for training a sequential sampler of discrete objects under an unnormalized target density and have been successfully used for various probabilistic modeling tasks. Existing training objectives for GFlowNets are either local to states or transitions, or propagate a reward signal over an entire sampling trajectory. We argue that these alternatives represent opposite ends of a gradient bias-variance tradeoff and propose a way to exploit this tradeoff to mitigate its harmful effects. Inspired by the TD(λ) algorithm in reinforcement learning, we introduce subtrajectory balance or SubTB(λ), a GFlowNet training objective that can learn from partial action subsequences of varying lengths. We show that SubTB(λ) accelerates sampler convergence in previously studied and new environments and enables training GFlowNets in environments with longer action sequences and sparser reward landscapes than what was possible before. We also perform a comparative analysis of stochastic gradient dynamics, shedding light on the bias-variance tradeoff in GFlowNet training and the advantages of subtrajectory balance.

Key facts. Cited by 132 papers in the map (data through 14 Oct 2026). Builds on 24 of its 53 references that are in the map. First posted on arXiv 26 Sep 2022.

Builds on

24 of 53 references.

References · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
  6. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  7. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 652
  8. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 392
  9. 2021GFlowNet Foundationscited by 290
  10. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 251
  11. 2018Provably Efficient Maximum Entropy Explorationcited by 242
  12. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 182
12 more
  1. 2018Deep Reinforcement Learning and the Deadly Triadcited by 174
  2. 2019Soft Actor-Critic for Discrete Action Settingscited by 174
  3. 2022Biological Sequence Design with GFlowNetscited by 174
  4. 2022Bayesian Structure Learning with Generative Flow Networkscited by 153
  5. 2021MARS: Markov Molecular Sampling for Multi-objective Drug Discoverycited by 125
  6. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 114
  7. 2019Conditioning by adaptive sampling for robust designcited by 111
  8. 2022Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimizationcited by 97
  9. 2020AdaLead: A simple and robust adaptive greedy search algorithm for sequence designcited by 65
  10. 2020Interference and Generalization in Temporal Difference Learningcited by 42
  11. 2019Deep Residual Reinforcement Learningcited by 21
  12. 2019Marginalized State Distribution Entropy Regularization in Policy Optimizationcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 31; 2024: 36; 2025: 30; 2026: 31.

132 papers.

Sort:

By area:

  1. 2022GFlowNets and variational inferencecited by 106
  2. 2023A theory of continuous generative flow networkscited by 101
  3. 2023Amortizing intractable inference in large language modelscited by 97
  4. 2023Better Training of GFlowNets with Local Credit and Incomplete Trajectoriescited by 85
  5. 2023Towards Understanding and Improving GFlowNet Trainingcited by 75
  6. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 72
  7. 2023Generative Flow Networks as Entropy-Regularized RLcited by 64
  8. 2023GFlowNets for AI-Driven Scientific Discoverycited by 63
  9. 2023Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimizationcited by 57
  10. 2024Ant Colony Sampling with GFlowNets for Combinatorial Optimizationcited by 53
  11. 2023Sample-efficient Multi-objective Molecular Optimization with GFlowNetscited by 50
  12. 2023Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Networkcited by 50
38 more
  1. 2023Local Search GFlowNetscited by 48
  2. 2024Improved off-policy training of diffusion samplerscited by 48
  3. 2024Sequential Controlled Langevin Diffusionscited by 48
  4. 2024Discrete Probabilistic Inference as Control in Multi-path Environmentscited by 44
  5. 2023GFlowNet-EM for learning compositional latent variable modelscited by 40
  6. 2023Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNetscited by 40
  7. 2025FlowRL: Matching Reward Distributions for LLM Reasoningcited by 40
  8. 2022A Variational Perspective on Generative Flow Networkscited by 39
  9. 2024Genetic-guided GFlowNets for Sample Efficient Molecular Optimizationcited by 39
  10. 2023Stochastic Generative Flow Networkscited by 32
  11. 2023Thompson sampling for improved exploration in GFlowNetscited by 32
  12. 2024Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Predictioncited by 32
  13. 2022Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayescited by 30
  14. 2023Maximum entropy GFlowNets with soft Q-learningcited by 30
  15. 2025Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Trainingcited by 29
  16. 2023Distributional GFlowNets with Quantile Flowscited by 28
  17. 2023DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNetscited by 27
  18. 2023Learning to Scale Logits for Temperature-Conditional GFlowNetscited by 27
  19. 2024Improving GFlowNets for Text-to-Image Diffusion Alignmentcited by 27
  20. 2023Learning Energy Decompositions for Partial Inference of GFlowNetscited by 24
  21. 2024Adaptive teachers for amortized samplerscited by 23
  22. 2022GFlowOut: Dropout with Generative Flow Networkscited by 22
  23. 2025From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster trainingcited by 19
  24. 2023An Empirical Study of the Effectiveness of Using a Replay Buffer on Mode Discovery in GFlowNetscited by 18
  25. 2024QGFN: Controllable Greediness with Action Valuescited by 18
  26. 2023Goal-conditioned GFlowNets for Controllable Multi-Objective Molecular Designcited by 17
  27. 2023Pre-Training and Fine-Tuning Generative Flow Networkscited by 17
  28. 2024Sampling from Energy-based Policies using Diffusioncited by 17
  29. 2025Offline Model-Based Optimization: Comprehensive Reviewcited by 15
  30. 2024Pessimistic Backward Policy for GFlowNetscited by 14
  31. 2024Optimizing Backward Policies in GFlowNets via Trajectory Likelihood Maximizationcited by 14
  32. 2023Order-Preserving GFlowNetscited by 13
  33. 2024Improved Off-policy Reinforcement Learning in Biological Sequence Designcited by 13
  34. 2023torchgfn: A PyTorch GFlowNet librarycited by 11
  35. 2024Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networkscited by 11
  36. 2025Revisiting Non-Acyclic GFlowNets in Discrete Environmentscited by 10
  37. 2025On scalable and efficient training of diffusion samplerscited by 10
  38. 2024Latent Logic Tree Extraction for Event Sequence Explanation from LLMscited by 9

50 most cited of 132 All 132 (JSON)