mlsanityData to 7 Oct 2026

Trajectory balance: Improved credit assignment in GFlowNets

arXiv 2201.13259 ↗First posted 31 Jan 2022cs.LG stat.MLcited by 244

Not on the map. Why?

Cited by 34 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generative flow networks (GFlowNets) are a method for learning a stochastic policy for generating compositional objects, such as graphs or strings, from a given unnormalized density by sequences of actions, where many possible action sequences may lead to the same object. We find previously proposed learning objectives for GFlowNets, flow matching and detailed balance, which are analogous to temporal difference learning, to be prone to inefficient credit propagation across long action sequences. We thus propose a new learning objective for GFlowNets, trajectory balance, as a more efficient alternative to previously used objectives. We prove that any global minimizer of the trajectory balance objective can define a policy that samples exactly from the target distribution. In experiments on four distinct domains, we empirically demonstrate the benefits of the trajectory balance objective for GFlowNet convergence, diversity of generated samples, and robustness to long action sequences and large action spaces.

Key facts. Cited by 244 papers in the map (data through 7 Oct 2026). Builds on 18 of its 44 references that are in the map. First posted on arXiv 31 Jan 2022.

Builds on

18 of 44 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  5. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  6. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 647
  7. 2020Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classificationcited by 500
  8. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  9. 2021GFlowNet Foundationscited by 286
  10. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 178
  11. 2018Deep Reinforcement Learning and the Deadly Triadcited by 170
  12. 2019Soft Actor-Critic for Discrete Action Settingscited by 161
6 more
  1. 2020Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learningcited by 134
  2. 2021MARS: Markov Molecular Sampling for Multi-objective Drug Discoverycited by 125
  3. 2014Neural Variational Inference and Learning in Belief Networkscited by 116
  4. 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
  5. 2020Interference and Generalization in Temporal Difference Learningcited by 41
  6. 2020Learning Discrete Energy-based Models via Auxiliary-variable Local Explorationcited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 14; 2023: 51; 2024: 59; 2025: 61; 2026: 59.

244 papers.

Sort:

By area:

  1. 2024Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Controlcited by 222
  2. 2022Biological Sequence Design with GFlowNetscited by 172
  3. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 129
  4. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 113
  5. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 105
  6. 2022GFlowNets and variational inferencecited by 103
  7. 2023Improved sampling via learned diffusionscited by 103
  8. 2022Multi-Objective GFlowNetscited by 100
  9. 2023A theory of continuous generative flow networkscited by 98
  10. 2023Amortizing intractable inference in large language modelscited by 94
  11. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 92
  12. 2023Better Training of GFlowNets with Local Credit and Incomplete Trajectoriescited by 83
38 more
  1. 2025Adjoint Sampling: Highly Scalable Diffusion Samplers via Adjoint Matchingcited by 78
  2. 2023Towards Understanding and Improving GFlowNet Trainingcited by 74
  3. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 70
  4. 2023Robust Scheduling with GFlowNetscited by 67
  5. 2023GFlowNets for AI-Driven Scientific Discoverycited by 63
  6. 2023Generative Flow Networks as Entropy-Regularized RLcited by 60
  7. 2024Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Samplingcited by 55
  8. 2024Learning diverse attacks on large language models for robust red-teaming and safety tuningcited by 54
  9. 2024Ant Colony Sampling with GFlowNets for Combinatorial Optimizationcited by 53
  10. 2022A-NeSI: A Scalable Approximate Method for Probabilistic Neurosymbolic Inferencecited by 50
  11. 2023Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Networkcited by 50
  12. 2022Generative Augmented Flow Networkscited by 48
  13. 2023Sample-efficient Multi-objective Molecular Optimization with GFlowNetscited by 48
  14. 2024Improved off-policy training of diffusion samplerscited by 47
  15. 2023Local Search GFlowNetscited by 45
  16. 2024DEFT: Efficient Fine-Tuning of Diffusion Models by Learning the Generalised h-transformcited by 44
  17. 2023GFlowNet-EM for learning compositional latent variable modelscited by 40
  18. 2023Let the Flows Tell: Solving Graph Combinatorial Optimization Problems with GFlowNetscited by 40
  19. 2024Discrete Probabilistic Inference as Control in Multi-path Environmentscited by 40
  20. 2022A Variational Perspective on Generative Flow Networkscited by 39
  21. 2024RGFN: Synthesizable Molecular Generation Using GFlowNetscited by 39
  22. 2025FlowRL: Matching Reward Distributions for LLM Reasoningcited by 38
  23. 2024Genetic-guided GFlowNets for Sample Efficient Molecular Optimizationcited by 36
  24. 2023Stochastic Generative Flow Networkscited by 32
  25. 2023Thompson sampling for improved exploration in GFlowNetscited by 31
  26. 2022Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayescited by 30
  27. 2023Maximum entropy GFlowNets with soft Q-learningcited by 30
  28. 2024Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNetscited by 30
  29. 2023Crystal-GFN: sampling crystals with desirable properties and constraintscited by 29
  30. 2023Distributional GFlowNets with Quantile Flowscited by 28
  31. 2024Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Predictioncited by 28
  32. 2023DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNetscited by 27
  33. 2023DAG Matters! GFlowNets Enhanced Explainer For Graph Neural Networkscited by 27
  34. 2024Improving GFlowNets for Text-to-Image Diffusion Alignmentcited by 27
  35. 2025Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Trainingcited by 27
  36. 2023Compositional Sculpting of Iterative Generative Processescited by 26
  37. 2023Learning to Scale Logits for Temperature-Conditional GFlowNetscited by 26
  38. 2024SynFlowNet: Design of Diverse and Novel Molecules with Synthesis Constraintscited by 26

50 most cited of 244 All 244 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025A Theory of Multi-Agent Generative Flow Networks
  2. 2025EMERGENT: Efficient and Manipulation-resistant Matching using GFlowNets
  3. 2026Universality and Convergence of Generative Flows
  4. 2025AlphaFlow: Understanding and Improving MeanFlow Models
  5. 2026Path-Guided Flow Matching for Dataset Distillation
  6. 2025A Theoretical Analysis of Discrete Flow Matching Generative Models
  7. 2024Stable Autonomous Flow Matching
  8. 2026Stepwise Credit Assignment for GRPO on Flow-Matching Models