mlsanityData to 7 Oct 2026

GFlowNet Fine-tuning for Diverse Correct Solutions in Mathematical Reasoning Tasks

arXiv 2410.20147 ↗First posted 26 Oct 2024cs.LGcited by 7

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Mathematical reasoning problems are among the most challenging, as they typically require an understanding of fundamental laws to solve. The laws are universal, but the derivation of the final answer changes depending on how a problem is approached. When training large language models (LLMs), learning the capability of generating such multiple solutions is essential to accelerate their use in mathematical education. To this end, we train LLMs using generative flow network (GFlowNet). Different from reward-maximizing reinforcement learning (RL), GFlowNet fine-tuning seeks to find diverse solutions by training the LLM whose distribution is proportional to a reward function. In numerical experiments, we evaluate GFlowNet fine-tuning and reward-maximizing RL in terms of accuracy and diversity. The results show that GFlowNet fine-tuning derives correct final answers from diverse intermediate reasoning steps, indicating the improvement of the capability of alternative solution generation.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 27 of its 35 references that are in the map. First posted on arXiv 26 Oct 2024.

Builds on

27 of 35 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Models3 Aug
  2. 2022Solving Quantitative Reasoning Problems with Language Models29 Jun

Other references · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2023Let's Verify Step by Stepcited by 3,598
  11. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  12. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
13 more
  1. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 724
  2. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  3. 2021GFlowNet Foundationscited by 286
  4. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  5. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 236
  6. 2022Biological Sequence Design with GFlowNetscited by 172
  7. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 129
  8. 2023DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scalescited by 107
  9. 2022Multi-Objective GFlowNetscited by 100
  10. 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 95
  11. 2023Amortizing intractable inference in large language modelscited by 94
  12. 2024MathVC: An LLM-Simulated Multi-Character Virtual Classroom for Mathematics Educationcited by 35
  13. 2024Weak-to-Strong Reasoningcited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025GFlowVLM: Enhancing Multi-step Reasoning in Vision-Language Models with Generative Flow Networkscited by 15
  2. 2025Accurate and Diverse LLM Mathematical Reasoning via Automated PRM-Guided GFlowNetscited by 10
  3. 2025Loss-Guided Auxiliary Agents for Overcoming Mode Collapse in GFlowNetscited by 4
  4. 2025GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networkscited by 3
  5. 2026Latent Thought Flow: Efficient Latent Reasoning in Large Language Modelscited by 3
  6. 2025Interpretable epistemic uncertainty decomposition in sequential generative models via polynomial chaos surrogatescited by 0
  7. 2026Stable-GFlowNet: Toward Diverse and Robust LLM Red-Teaming via Contrastive Trajectory Balancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Proof Flow: Preliminary Study on Generative Flow Network Language Model Tuning for Formal Reasoning
  2. 2024Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examples
  3. 2026GFlowRL: Scaling Distribution-Matching RL to Large Language Models
  4. 2025Does Learning Mathematical Problem-Solving Generalize to Broader Reasoning?
  5. 2025A Survey on Large Language Models for Mathematical Reasoning
  6. 2025Does Math Reasoning Improve General LLM Capabilities? Understanding Transferability of LLM Reasoning
  7. 2025LogicPuzzleRL: Cultivating Robust Mathematical Reasoning in LLMs via Reinforcement Learning
  8. 2025A Survey on Mathematical Reasoning and Optimization with Large Language Models