mlsanityData to 7 Oct 2026

GFlowNets with Human Feedback

arXiv 2305.07036 ↗First posted 11 May 2023cs.LG cs.AIcited by 3

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

We propose the GFlowNets with Human Feedback (GFlowHF) framework to improve the exploration ability when training AI models. For tasks where the reward is unknown, we fit the reward function through human evaluations on different trajectories. The goal of GFlowHF is to learn a policy that is strictly proportional to human ratings, instead of only focusing on human favorite ratings like RLHF. Experiments show that GFlowHF can achieve better exploration ability than RLHF.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 17 of its 18 references that are in the map. First posted on arXiv 11 May 2023.

Builds on

17 of 18 references.

Closest ancestors · 3

Same direction, newest first

  1. 2021WebGPT: Browser-assisted question-answering with human feedback17 Dec
  2. 2019Fine-Tuning Language Models from Human Preferences18 Sep
  3. 2017Deep reinforcement learning from human preferences12 Jun

Other references · 14

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2020Learning to summarize from human feedbackcited by 2,562
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  7. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  8. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  9. 2022Bayesian Structure Learning with Generative Flow Networkscited by 153
  10. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 113
  11. 2023DAG Matters! GFlowNets Enhanced Explainer For Graph Neural Networkscited by 27
  12. 2023CFlowNets: Continuous Control with Generative Flow Networkscited by 19
2 more
  1. 2022GFlowCausal: Generative Flow Networks for Causal Discoverycited by 11
  2. 2023Generative Flow Networks for Precise Reward-Oriented Active Learning on Graphscited by 3

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 2.

3 papers.

  1. 2023Expert-Aided Causal Discovery of Ancestral Graphscited by 8
  2. 2024Random Policy Evaluation Uncovers Policies of Generative Flow Networkscited by 5
  3. 2024Bifurcated Generative Flow Networkscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Training
  2. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
  3. 2024RLHF Workflow: From Reward Modeling to Online RLHF
  4. 2023A Survey of Reinforcement Learning from Human Feedback
  5. 2025Enhancing RLHF with Human Gaze Modeling
  6. 2025G-Core: A Simple, Scalable and Balanced RLHF Trainer
  7. 2024RLHF from Heterogeneous Feedback via Personalization and Preference Aggregation
  8. 2024Uni-RLHF: Universal Platform and Benchmark Suite for Reinforcement Learning with Diverse Human Feedback