mlsanityData to 7 Oct 2026

Generative Flow Networks as Entropy-Regularized RL

arXiv 2310.12934 ↗First posted 19 Oct 2023cs.LG stat.MLcited by 60

Not on the map. Why?

Abstract (arXiv)

The recently proposed generative flow networks (GFlowNets) are a method of training a policy to sample compositional discrete objects with probabilities proportional to a given reward via a sequence of actions. GFlowNets exploit the sequential nature of the problem, drawing parallels with reinforcement learning (RL). Our work extends the connection between RL and GFlowNets to a general case. We demonstrate how the task of learning a generative flow network can be efficiently redefined as an entropy-regularized RL problem with a specific reward and regularizer structure. Furthermore, we illustrate the practical efficiency of this reformulation by applying standard soft RL algorithms to GFlowNet training across several probabilistic modeling tasks. Contrary to previously reported results, we show that entropic RL approaches can be competitive against established GFlowNet training methods. This perspective opens a direct path for integrating RL principles into the realm of generative flow networks.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Builds on 42 of its 64 references that are in the map. First posted on arXiv 19 Oct 2023.

Builds on

42 of 64 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  6. 2015Continuous control with deep reinforcement learningcited by 3,279
  7. 2015Prioritized Experience Replaycited by 1,070
  8. 2018Exploration by Random Network Distillationcited by 966
  9. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  10. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  11. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 647
  12. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
30 more
  1. 2016Deep Exploration via Bootstrapped DQNcited by 518
  2. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  3. 2021GFlowNet Foundationscited by 286
  4. 2019A Theory of Regularized Markov Decision Processescited by 279
  5. 2017Count-Based Exploration with Neural Density Modelscited by 255
  6. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  7. 2017A unified view of entropy-regularized Markov decision processescited by 174
  8. 2017Equivalence Between Policy Gradients and Soft Q-Learningcited by 172
  9. 2019Soft Actor-Critic for Discrete Action Settingscited by 161
  10. 2022Bayesian Structure Learning with Generative Flow Networkscited by 153
  11. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 129
  12. 2015Taming the Noise in Reinforcement Learning via Soft Updatescited by 125
  13. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 113
  14. 2022GFlowNets and variational inferencecited by 103
  15. 2023A theory of continuous generative flow networkscited by 98
  16. 2021Reward is enough for convex MDPscited by 82
  17. 2023Towards Understanding and Improving GFlowNet Trainingcited by 74
  18. 2023TorchRL: A data-driven decision-making library for PyTorchcited by 59
  19. 2022Generative Augmented Flow Networkscited by 48
  20. 2022A Variational Perspective on Generative Flow Networkscited by 39
  21. 2023Fast Rates for Maximum Entropy Explorationcited by 32
  22. 2023Thompson sampling for improved exploration in GFlowNetscited by 31
  23. 2023Distributional GFlowNets with Quantile Flowscited by 28
  24. 2022Unifying Generative Models with GFlowNets and Beyondcited by 24
  25. 2013Efficient Reinforcement Learning in Deterministic Systems with Value Function Generalizationcited by 19
  26. 2023An Empirical Study of the Effectiveness of Using a Replay Buffer on Mode Discovery in GFlowNetscited by 17
  27. 2023Generative Flow Networks: a Markov Chain Perspectivecited by 12
  28. 2023VA-learning as a more efficient alternative to Q-learningcited by 11
  29. 2023torchgfn: A PyTorch GFlowNet librarycited by 9
  30. 2023Unlocking the Power of Representations in Long-term Novelty-based Explorationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 21; 2025: 23; 2026: 15.

60 papers.

Sort:

By area:

  1. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 92
  2. 2024Improved off-policy training of diffusion samplerscited by 47
  3. 2024Discrete Probabilistic Inference as Control in Multi-path Environmentscited by 40
  4. 2025FlowRL: Matching Reward Distributions for LLM Reasoningcited by 38
  5. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 31
  6. 2023Maximum entropy GFlowNets with soft Q-learningcited by 30
  7. 2024Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNetscited by 30
  8. 2025KL-Regularized Reinforcement Learning is Designed to Mode Collapsecited by 29
  9. 2024Improving GFlowNets for Text-to-Image Diffusion Alignmentcited by 27
  10. 2024Adaptive teachers for amortized samplerscited by 23
  11. 2025LeMat-GenBench: A Unified Evaluation Framework for Crystal Generative Modelscited by 20
  12. 2024QGFN: Controllable Greediness with Action Valuescited by 18
38 more
  1. 2025From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster trainingcited by 17
  2. 2025Learning to Sample Effective and Diverse Prompts for Text-to-Image Generationcited by 16
  3. 2024Sampling from Energy-based Policies using Diffusioncited by 15
  4. 2024Pessimistic Backward Policy for GFlowNetscited by 13
  5. 2024Improved Off-policy Reinforcement Learning in Biological Sequence Designcited by 13
  6. 2024Optimizing Backward Policies in GFlowNets via Trajectory Likelihood Maximizationcited by 12
  7. 2026Future-KL Regularized GRPO: Process-Level Credit Assignment from f-Divergence Regularizationcited by 11
  8. 2025Accurate and Diverse LLM Mathematical Reasoning via Automated PRM-Guided GFlowNetscited by 10
  9. 2024Latent Logic Tree Extraction for Event Sequence Explanation from LLMscited by 9
  10. 2024Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networkscited by 9
  11. 2025Group-Aware Reinforcement Learning for Output Diversity in Large Language Modelscited by 9
  12. 2024Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNetscited by 8
  13. 2024Improving GFlowNets with Monte Carlo Tree Searchcited by 8
  14. 2025Revisiting Non-Acyclic GFlowNets in Discrete Environmentscited by 8
  15. 2025Autoregressive Language Models are Secretly Energy-Based Models: Insights into the Lookahead Capabilities of Next-Token Predictioncited by 7
  16. 2024Investigating Generalization Behaviours of Generative Flow Networkscited by 6
  17. 2024GDPO: Learning to Directly Align Language Models with Diversity Using GFlowNetscited by 6
  18. 2025Adaptive Destruction Processes for Diffusion Samplerscited by 6
  19. 2024Random Policy Evaluation Uncovers Policies of Generative Flow Networkscited by 5
  20. 2025D2PPO: Diffusion Policy Policy Optimization with Dispersive Losscited by 5
  21. 2025Reinforced sequential Monte Carlo for amortised samplingcited by 5
  22. 2025Symmetry-Aware GFlowNetscited by 4
  23. 2024On Generalization for Generative Flow Networkscited by 3
  24. 2024Action abstractions for amortized samplingcited by 3
  25. 2025Relative Trajectory Balance is equivalent to Trust-PCLcited by 3
  26. 2025FlowQ-Net: A Generative Framework for Automated Quantum Circuit Designcited by 3
  27. 2024ETGL-DDPG: A Deep Deterministic Policy Gradient Algorithm for Sparse Reward Continuous Controlcited by 2
  28. 2025Learning Decision Trees as Amortized Structure Inferencecited by 2
  29. 2025Discrete Compositional Generation via General Soft Operators and Robust Reinforcement Learningcited by 2
  30. 2025Mitigating Hallucinations in LM-Based TTS Models via Distribution Alignment Using GFlowNetscited by 2
  31. 2025gfnx: Fast and Scalable Library for Generative Flow Networks in JAXcited by 2
  32. 2026Discrete diffusion samplers and bridges: Off-policy algorithms and applications in latent spacescited by 2
  33. 2026Rooted Absorbed Prefix Trajectory Balance with Submodular Replay for GFlowNet Trainingcited by 2
  34. 2026Learning Shortest Paths with Generative Flow Networkscited by 2
  35. 2025Consistent Amortized Clustering via Generative Flow Networkscited by 1
  36. 2026Evaluating GFlowNet from partial episodes for stable and flexible policy-based trainingcited by 1
  37. 2026GFlowRL: Scaling Distribution-Matching RL to Large Language Modelscited by 1
  38. 2025Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Trainingcited by 0

50 most cited of 60 All 60 (JSON)