# Pessimistic Backward Policy for GFlowNets

arXiv: https://arxiv.org/abs/2405.16012 · HTML: https://mlsanity.com/paper/2405.16012/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 41 references that are in the map.
- First posted on arXiv 25 May 2024.

## Builds on (most cited first, 30 of 31)

- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation](https://mlsanity.com/paper/2106.04399/) (2021)
- [GFlowNet Foundations](https://mlsanity.com/paper/2111.09266/) (2021)
- [Trajectory balance: Improved credit assignment in GFlowNets](https://mlsanity.com/paper/2201.13259/) (2022)
- [Biological Sequence Design with GFlowNets](https://mlsanity.com/paper/2203.04115/) (2022)
- [Bayesian Structure Learning with Generative Flow Networks](https://mlsanity.com/paper/2202.13903/) (2022)
- [Learning GFlowNets from partial episodes for improved convergence and stability](https://mlsanity.com/paper/2209.12782/) (2022)
- [Generative Flow Networks for Discrete Probabilistic Modeling](https://mlsanity.com/paper/2202.01361/) (2022)
- [GFlowNets and variational inference](https://mlsanity.com/paper/2210.00580/) (2022)
- [Multi-Objective GFlowNets](https://mlsanity.com/paper/2210.12765/) (2022)
- [A theory of continuous generative flow networks](https://mlsanity.com/paper/2301.12594/) (2023)
- [Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization](https://mlsanity.com/paper/2202.08450/) (2022)
- [Amortizing intractable inference in large language models](https://mlsanity.com/paper/2310.04363/) (2023)
- [Better Training of GFlowNets with Local Credit and Incomplete Trajectories](https://mlsanity.com/paper/2302.01687/) (2023)
- [Towards Understanding and Improving GFlowNet Training](https://mlsanity.com/paper/2305.07170/) (2023)
- [Robust Scheduling with GFlowNets](https://mlsanity.com/paper/2302.05446/) (2023)
- [AdaLead: A simple and robust adaptive greedy search algorithm for sequence design](https://mlsanity.com/paper/2010.02141/) (2020)
- [Generative Flow Networks as Entropy-Regularized RL](https://mlsanity.com/paper/2310.12934/) (2023)
- [Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimization](https://mlsanity.com/paper/2310.02679/) (2023)
- [Ant Colony Sampling with GFlowNets for Combinatorial Optimization](https://mlsanity.com/paper/2403.07041/) (2024)
- [Generative Augmented Flow Networks](https://mlsanity.com/paper/2210.03308/) (2022)
- [Sample-efficient Multi-objective Molecular Optimization with GFlowNets](https://mlsanity.com/paper/2302.04040/) (2023)
- [Local Search GFlowNets](https://mlsanity.com/paper/2310.02710/) (2023)
- [Discrete Probabilistic Inference as Control in Multi-path Environments](https://mlsanity.com/paper/2402.10309/) (2024)
- [Thompson sampling for improved exploration in GFlowNets](https://mlsanity.com/paper/2306.17693/) (2023)
- [Maximum entropy GFlowNets with soft Q-learning](https://mlsanity.com/paper/2312.14331/) (2023)
- [Learning to Scale Logits for Temperature-Conditional GFlowNets](https://mlsanity.com/paper/2310.02823/) (2023)
- [Learning Energy Decompositions for Partial Inference of GFlowNets](https://mlsanity.com/paper/2310.03301/) (2023)
- [QGFN: Controllable Greediness with Action Values](https://mlsanity.com/paper/2402.05234/) (2024)

## Built on by (most cited first, 14 of 14)

- [Adaptive teachers for amortized samplers](https://mlsanity.com/paper/2410.01432/) (2024)
- [Optimizing Backward Policies in GFlowNets via Trajectory Likelihood Maximization](https://mlsanity.com/paper/2410.15474/) (2024)
- [Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networks](https://mlsanity.com/paper/2410.02596/) (2024)
- [Revisiting Non-Acyclic GFlowNets in Discrete Environments](https://mlsanity.com/paper/2502.07735/) (2025)
- [Adaptive Destruction Processes for Diffusion Samplers](https://mlsanity.com/paper/2506.01541/) (2025)
- [Scalable and Cost-Efficient de Novo Template-Based Molecular Generation](https://mlsanity.com/paper/2506.19865/) (2025)
- [Learning Shortest Paths with Generative Flow Networks](https://mlsanity.com/paper/2603.01786/) (2026)
- [Evaluating GFlowNet from partial episodes for stable and flexible policy-based training](https://mlsanity.com/paper/2603.01047/) (2026)
- [Stable GFlowNets with TV Monitoring and Probabilistic Guarantees](https://mlsanity.com/paper/2605.01729/) (2026)
- [Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Training](https://mlsanity.com/paper/2505.20110/) (2025)
- [Flow of Spans: Generalizing Language Models to Dynamic Span-Vocabulary via GFlowNets](https://mlsanity.com/paper/2602.10583/) (2026)
- [A Systematic Evaluation of Co-folding Model Representations for Small-Molecule Learning](https://mlsanity.com/paper/2602.13249/) (2026)
- [Your GFlowNet Secretly Learns an Optimal Transport Plan](https://mlsanity.com/paper/2606.06272/) (2026)
- [Towards Understanding the Training of GFlowNets: A Theoretical Perspective](https://mlsanity.com/paper/2610.13711/) (2026)

## Abstract (arXiv)

This paper studies Generative Flow Networks (GFlowNets), which learn to sample objects proportionally to a given reward function through the trajectory of state transitions. In this work, we observe that GFlowNets tend to under-exploit the high-reward objects due to training on insufficient number of trajectories, which may lead to a large gap between the estimated flow and the (known) reward value. In response to this challenge, we propose a pessimistic backward policy for GFlowNets (PBP-GFN), which maximizes the observed flow to align closely with the true reward for the object. We extensively evaluate PBP-GFN across eight benchmarks, including hyper-grid environment, bag generation, structured set generation, molecular generation, and four RNA sequence generation tasks. In particular, PBP-GFN enhances the discovery of high-reward objects, maintains the diversity of the objects, and consistently outperforms existing methods.