mlsanityData to 7 Oct 2026

Maximum entropy GFlowNets with soft Q-learning

arXiv 2312.14331 ↗First posted 21 Dec 2023cs.LGcited by 30

Not on the map. Why?

Abstract (arXiv)

Generative Flow Networks (GFNs) have emerged as a powerful tool for sampling discrete objects from unnormalized distributions, offering a scalable alternative to Markov Chain Monte Carlo (MCMC) methods. While GFNs draw inspiration from maximum entropy reinforcement learning (RL), the connection between the two has largely been unclear and seemingly applicable only in specific cases. This paper addresses the connection by constructing an appropriate reward function, thereby establishing an exact relationship between GFNs and maximum entropy RL. This construction allows us to introduce maximum entropy GFNs, which, in contrast to GFNs with uniform backward policy, achieve the maximum entropy attainable by GFNs without constraints on the state space.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 24 of its 56 references that are in the map. First posted on arXiv 21 Dec 2023.

Builds on

24 of 56 references.

References · 24

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  3. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 647
  4. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  5. 2021GFlowNet Foundationscited by 286
  6. 2019A Theory of Regularized Markov Decision Processescited by 279
  7. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  8. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 178
  9. 2022Bayesian Structure Learning with Generative Flow Networkscited by 153
  10. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 129
  11. 2015Taming the Noise in Reinforcement Learning via Soft Updatescited by 125
  12. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 113
12 more
  1. 2023Extreme Q-Learning: MaxEnt RL without Entropycited by 113
  2. 2022Multi-Objective GFlowNetscited by 100
  3. 2023Towards Understanding and Improving GFlowNet Trainingcited by 74
  4. 2020Munchausen Reinforcement Learningcited by 68
  5. 2023Generative Flow Networks as Entropy-Regularized RLcited by 60
  6. 2018Differentiable Dynamic Programming for Structured Prediction and Attentioncited by 58
  7. 2021Twice regularized MDPs and the equivalence between robustness and regularizationcited by 46
  8. 2023Local Search GFlowNetscited by 45
  9. 2023Stochastic Generative Flow Networkscited by 32
  10. 2013Credal Networks under Maximum Entropycited by 13
  11. 2023Expected flow networks in stochastic environments and two-player zero-sum gamescited by 7
  12. 2019Approximate Inference in Discrete Distributions with Monte Carlo Tree Search and Value Functionscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 12; 2026: 3.

30 papers.

Sort:

By area:

  1. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 92
  2. 2024Ant Colony Sampling with GFlowNets for Combinatorial Optimizationcited by 53
  3. 2024Discrete Probabilistic Inference as Control in Multi-path Environmentscited by 40
  4. 2025FlowRL: Matching Reward Distributions for LLM Reasoningcited by 38
  5. 2024Genetic-guided GFlowNets for Sample Efficient Molecular Optimizationcited by 36
  6. 2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 31
  7. 2024Improving GFlowNets for Text-to-Image Diffusion Alignmentcited by 27
  8. 2024SynFlowNet: Design of Diverse and Novel Molecules with Synthesis Constraintscited by 26
  9. 2024QGFN: Controllable Greediness with Action Valuescited by 18
  10. 2024Pessimistic Backward Policy for GFlowNetscited by 13
  11. 2024Optimizing Backward Policies in GFlowNets via Trajectory Likelihood Maximizationcited by 12
  12. 2025RL-finetuning LLMs from on- and off-policy data with a single algorithmcited by 11
18 more
  1. 2024Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networkscited by 9
  2. 2024Improving GFlowNets with Monte Carlo Tree Searchcited by 8
  3. 2025Revisiting Non-Acyclic GFlowNets in Discrete Environmentscited by 8
  4. 2024Investigating Generalization Behaviours of Generative Flow Networkscited by 6
  5. 2024GDPO: Learning to Directly Align Language Models with Diversity Using GFlowNetscited by 6
  6. 2025Adaptive Destruction Processes for Diffusion Samplerscited by 6
  7. 2024Random Policy Evaluation Uncovers Policies of Generative Flow Networkscited by 5
  8. 2025Scalable and Cost-Efficient de Novo Template-Based Molecular Generationcited by 5
  9. 2025Reinforced sequential Monte Carlo for amortised samplingcited by 5
  10. 2025Symmetry-Aware GFlowNetscited by 4
  11. 2024Action abstractions for amortized samplingcited by 3
  12. 2025Relative Trajectory Balance is equivalent to Trust-PCLcited by 3
  13. 2025Discrete Compositional Generation via General Soft Operators and Robust Reinforcement Learningcited by 2
  14. 2025gfnx: Fast and Scalable Library for Generative Flow Networks in JAXcited by 2
  15. 2026GFlowRL: Scaling Distribution-Matching RL to Large Language Modelscited by 1
  16. 2025Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Trainingcited by 0
  17. 2026Controlling Exploration-Exploitation in GFlowNets via Markov Chain Perspectivescited by 0
  18. 2026Your GFlowNet Secretly Learns an Optimal Transport Plancited by 0