# Generative Flow Networks: a Markov Chain Perspective

arXiv: https://arxiv.org/abs/2307.01422 · HTML: https://mlsanity.com/paper/2307.01422/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 16 references that are in the map.
- First posted on arXiv 4 Jul 2023.

## Builds on (most cited first, 10 of 10)

- [Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation](https://mlsanity.com/paper/2106.04399/) (2021)
- [GFlowNet Foundations](https://mlsanity.com/paper/2111.09266/) (2021)
- [Trajectory balance: Improved credit assignment in GFlowNets](https://mlsanity.com/paper/2201.13259/) (2022)
- [Biological Sequence Design with GFlowNets](https://mlsanity.com/paper/2203.04115/) (2022)
- [Bayesian Structure Learning with Generative Flow Networks](https://mlsanity.com/paper/2202.13903/) (2022)
- [A theory of continuous generative flow networks](https://mlsanity.com/paper/2301.12594/) (2023)
- [GFlowNets for AI-Driven Scientific Discovery](https://mlsanity.com/paper/2302.00615/) (2023)
- [Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network](https://mlsanity.com/paper/2305.19366/) (2023)
- [GFlowNet-EM for learning compositional latent variable models](https://mlsanity.com/paper/2302.06576/) (2023)
- [CFlowNets: Continuous Control with Generative Flow Networks](https://mlsanity.com/paper/2303.02430/) (2023)

## Built on by (most cited first, 14 of 14)

- [Generative Flow Networks as Entropy-Regularized RL](https://mlsanity.com/paper/2310.12934/) (2023)
- [Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examples](https://mlsanity.com/paper/2406.05673/) (2024)
- [From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster training](https://mlsanity.com/paper/2501.06148/) (2025)
- [A Theory of Non-Acyclic Generative Flow Networks](https://mlsanity.com/paper/2312.15246/) (2023)
- [Random Policy Evaluation Uncovers Policies of Generative Flow Networks](https://mlsanity.com/paper/2406.02213/) (2024)
- [Secrets of GFlowNets' Learning Behavior: A Theoretical Study](https://mlsanity.com/paper/2505.02035/) (2025)
- [Enhancing Solution Efficiency in Reinforcement Learning: Leveraging Sub-GFlowNet and Entropy Integration](https://mlsanity.com/paper/2410.00461/) (2024)
- [Controlling Exploration-Exploitation in GFlowNets via Markov Chain Perspectives](https://mlsanity.com/paper/2602.01749/) (2026)
- [Stop the Sampler! Classifier-Based Adaptive Stopping for Sampling Kernels](https://mlsanity.com/paper/2606.16073/) (2026)
- [IFlowNets: Extending Generative Samplers to Learn Strategies in Incomplete Information Games](https://mlsanity.com/paper/2608.05422/) (2026)
- [Path-dependent Discrete Amortized Inference](https://mlsanity.com/paper/2608.08644/) (2026)
- [Particle GFlowNets: Rethinking Generative Marginalization Models](https://mlsanity.com/paper/2609.11538/) (2026)
- [Universality and Convergence of Generative Flows](https://mlsanity.com/paper/2610.05490/) (2026)
- [Learning a Mixture of GFlowNets](https://mlsanity.com/paper/2610.07562/) (2026)

## Abstract (arXiv)

While Markov chain Monte Carlo methods (MCMC) provide a general framework to sample from a probability distribution defined up to normalization, they often suffer from slow convergence to the target distribution when the latter is highly multi-modal. Recently, Generative Flow Networks (GFlowNets) have been proposed as an alternative framework to mitigate this issue when samples have a clear compositional structure, by treating sampling as a sequential decision making problem. Although they were initially introduced from the perspective of flow networks, the recent advances of GFlowNets draw more and more inspiration from the Markov chain literature, bypassing completely the need for flows. In this paper, we formalize this connection and offer a new perspective for GFlowNets using Markov chains, showing a unifying view for GFlowNets regardless of the nature of the state space as recurrent Markov chains. Positioning GFlowNets under the same theoretical framework as MCMC methods also allows us to identify the similarities between both frameworks, and most importantly to highlight their