# QGFN: Controllable Greediness with Action Values

arXiv: https://arxiv.org/abs/2402.05234 · HTML: https://mlsanity.com/paper/2402.05234/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 38 references that are in the map.
- First posted on arXiv 7 Feb 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [Flow Network based Generative Models for Non-Iterative Diverse Candidate Generation](https://mlsanity.com/paper/2106.04399/) (2021)
- [GFlowNet Foundations](https://mlsanity.com/paper/2111.09266/) (2021)
- [Trajectory balance: Improved credit assignment in GFlowNets](https://mlsanity.com/paper/2201.13259/) (2022)
- [Biological Sequence Design with GFlowNets](https://mlsanity.com/paper/2203.04115/) (2022)
- [Learning GFlowNets from partial episodes for improved convergence and stability](https://mlsanity.com/paper/2209.12782/) (2022)
- [GFlowNets and variational inference](https://mlsanity.com/paper/2210.00580/) (2022)
- [Multi-Objective GFlowNets](https://mlsanity.com/paper/2210.12765/) (2022)
- [A theory of continuous generative flow networks](https://mlsanity.com/paper/2301.12594/) (2023)
- [Better Training of GFlowNets with Local Credit and Incomplete Trajectories](https://mlsanity.com/paper/2302.01687/) (2023)
- [AdaLead: A simple and robust adaptive greedy search algorithm for sequence design](https://mlsanity.com/paper/2010.02141/) (2020)
- [Generative Flow Networks as Entropy-Regularized RL](https://mlsanity.com/paper/2310.12934/) (2023)
- [Local Search GFlowNets](https://mlsanity.com/paper/2310.02710/) (2023)
- [Molecular Mechanics-Driven Graph Neural Network with Multiplex Graph for Molecular Structures](https://mlsanity.com/paper/2011.07457/) (2020)
- [Stochastic Generative Flow Networks](https://mlsanity.com/paper/2302.09465/) (2023)
- [Thompson sampling for improved exploration in GFlowNets](https://mlsanity.com/paper/2306.17693/) (2023)
- [Maximum entropy GFlowNets with soft Q-learning](https://mlsanity.com/paper/2312.14331/) (2023)
- [Learning to Scale Logits for Temperature-Conditional GFlowNets](https://mlsanity.com/paper/2310.02823/) (2023)
- [Unifying Generative Models with GFlowNets and Beyond](https://mlsanity.com/paper/2209.02606/) (2022)
- [An Empirical Study of the Effectiveness of Using a Replay Buffer on Mode Discovery in GFlowNets](https://mlsanity.com/paper/2307.07674/) (2023)
- [DGFN: Double Generative Flow Networks](https://mlsanity.com/paper/2310.19685/) (2023)

## Built on by (most cited first, 18 of 18)

- [Learning diverse attacks on large language models for robust red-teaming and safety tuning](https://mlsanity.com/paper/2405.18540/) (2024)
- [Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNets](https://mlsanity.com/paper/2412.07775/) (2024)
- [Adaptive teachers for amortized samplers](https://mlsanity.com/paper/2410.01432/) (2024)
- [Learning to Sample Effective and Diverse Prompts for Text-to-Image Generation](https://mlsanity.com/paper/2502.11477/) (2025)
- [Pessimistic Backward Policy for GFlowNets](https://mlsanity.com/paper/2405.16012/) (2024)
- [Optimizing Backward Policies in GFlowNets via Trajectory Likelihood Maximization](https://mlsanity.com/paper/2410.15474/) (2024)
- [Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networks](https://mlsanity.com/paper/2410.02596/) (2024)
- [Revisiting Non-Acyclic GFlowNets in Discrete Environments](https://mlsanity.com/paper/2502.07735/) (2025)
- [Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNets](https://mlsanity.com/paper/2406.01150/) (2024)
- [Random Policy Evaluation Uncovers Policies of Generative Flow Networks](https://mlsanity.com/paper/2406.02213/) (2024)
- [Bifurcated Generative Flow Networks](https://mlsanity.com/paper/2406.01901/) (2024)
- [Exploring Multiple High-Scoring Subspaces in Generative Flow Networks](https://mlsanity.com/paper/2602.11491/) (2026)
- [Stable GFlowNets with TV Monitoring and Probabilistic Guarantees](https://mlsanity.com/paper/2605.01729/) (2026)
- [Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Training](https://mlsanity.com/paper/2505.20110/) (2025)
- [Planning-Augmented Sampling with Early Guidance for High-Reward Discovery](https://mlsanity.com/paper/2510.00805/) (2025)
- [Controlling Exploration-Exploitation in GFlowNets via Markov Chain Perspectives](https://mlsanity.com/paper/2602.01749/) (2026)
- [Your GFlowNet Secretly Learns an Optimal Transport Plan](https://mlsanity.com/paper/2606.06272/) (2026)
- [Beyond Drug Discovery: The Nanotechnology Molecular Optimization (NMO) Benchmark](https://mlsanity.com/paper/2606.30170/) (2026)

## Abstract (arXiv)

Generative Flow Networks (GFlowNets; GFNs) are a family of energy-based generative methods for combinatorial objects, capable of generating diverse and high-utility samples. However, consistently biasing GFNs towards producing high-utility samples is non-trivial. In this work, we leverage connections between GFNs and reinforcement learning (RL) and propose to combine the GFN policy with an action-value estimate, Q, to create greedier sampling policies which can be controlled by a mixing parameter. We show that several variants of the proposed method, QGFN, are able to improve on the number of high-reward samples generated in a variety of tasks without sacrificing diversity.