# Process-Supervised LLM Recommenders via Flow-guided Tuning

arXiv: https://arxiv.org/abs/2503.07377 · HTML: https://mlsanity.com/paper/2503.07377/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2025.

## Built on by (most cited first, 10 of 10)

- [Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimization](https://mlsanity.com/paper/2506.06964/) (2025)
- [In-context Ranking Preference Optimization](https://mlsanity.com/paper/2504.15477/) (2025)
- [Fine-grained List-wise Alignment for Generative Medication Recommendation](https://mlsanity.com/paper/2505.20218/) (2025)
- [IGD: Token Decisiveness Modeling via Information Gain in LLMs for Personalized Recommendation](https://mlsanity.com/paper/2506.13229/) (2025)
- [GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networks](https://mlsanity.com/paper/2506.16114/) (2025)
- [Don't Start Over: A Cost-Effective Framework for Migrating Personalized Prompts Between LLMs](https://mlsanity.com/paper/2601.12034/) (2026)
- [Understanding Reinforcement Learning for Model Training, and future directions with GRAPE](https://mlsanity.com/paper/2509.04501/) (2025)
- [ItemRAG: Item-Based Retrieval-Augmented Generation for LLM-Based Recommendation](https://mlsanity.com/paper/2511.15141/) (2025)
- [Position-Aware Drafting for Inference Acceleration in LLM-Based Generative List-Wise Recommendation](https://mlsanity.com/paper/2604.27747/) (2026)
- [LBR: Towards Mitigating Length Bias in Large Language Models for Recommendation](https://mlsanity.com/paper/2607.04270/) (2026)

## Abstract (arXiv)

While large language models (LLMs) are increasingly adapted for recommendation systems via supervised fine-tuning (SFT), this approach amplifies popularity bias due to its likelihood maximization objective, compromising recommendation diversity and fairness. To address this, we present Flow-guided fine-tuning recommender (Flower), which replaces SFT with a Generative Flow Network (GFlowNet) framework that enacts process supervision through token-level reward propagation. Flower's key innovation lies in decomposing item-level rewards into constituent token rewards, enabling direct alignment between token generation probabilities and their reward signals. This mechanism achieves three critical advancements: (1) popularity bias mitigation and fairness enhancement through empirical distribution matching, (2) preservation of diversity through GFlowNet's proportional sampling, and (3) flexible integration of personalized preferences via adaptable token rewards. Experiments demonstrate Flower's superior distribution-fitting capability and its significant advantages over traditional SFT in terms of accuracy, fairness, and diversity, highlighting its potential to improve LLM-based recommendation systems. The implementation is available via https://github.com/MrPeach0301/Flower