# Optimizing Generative Ranking Relevance via Reinforcement Learning in Xiaohongshu Search

arXiv: https://arxiv.org/abs/2512.00968 · HTML: https://mlsanity.com/paper/2512.00968/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 27 of its 49 references that are in the map.
- First posted on arXiv 30 Nov 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 27 of 27)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training](https://mlsanity.com/paper/2501.17161/) (2025)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Generative Verifiers: Reward Modeling as Next-Token Prediction](https://mlsanity.com/paper/2408.15240/) (2024)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [Large Language Models for Information Retrieval: A Survey](https://mlsanity.com/paper/2308.07107/) (2023)
- [Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting](https://mlsanity.com/paper/2306.17563/) (2023)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://mlsanity.com/paper/2310.08319/) (2023)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses](https://mlsanity.com/paper/2210.10634/) (2022)
- [What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secret](https://mlsanity.com/paper/2503.01491/) (2025)
- [Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labels](https://mlsanity.com/paper/2310.14122/) (2023)
- [Rank1: Test-Time Compute for Reranking in Information Retrieval](https://mlsanity.com/paper/2502.18418/) (2025)
- [Rank-R1: Enhancing Reasoning in LLM-based Document Rerankers via Reinforcement Learning](https://mlsanity.com/paper/2503.06034/) (2025)
- [Pre-trained Language Model based Ranking in Baidu Search](https://mlsanity.com/paper/2105.11108/) (2021)
- [JudgeRank: Leveraging Large Language Models for Reasoning-Intensive Reranking](https://mlsanity.com/paper/2411.00142/) (2024)
- [CAPO: Towards Enhancing LLM Reasoning through Generative Credit Assignment](https://mlsanity.com/paper/2508.02298/) (2025)
- [REARANK: Reasoning Re-ranking Agent via Reinforcement Learning](https://mlsanity.com/paper/2505.20046/) (2025)
- [DaRec: A Disentangled Alignment Framework for Large Language Model and Recommender System](https://mlsanity.com/paper/2408.08231/) (2024)
- [CTR-Guided Generative Query Suggestion in Conversational Search](https://mlsanity.com/paper/2507.04072/) (2025)
- [LREF: A Novel LLM-based Relevance Framework for E-commerce](https://mlsanity.com/paper/2503.09223/) (2025)
- [RedOne: Revealing Domain-specific LLM Post-Training in Social Networking Services](https://mlsanity.com/paper/2507.10605/) (2025)
- [An Axiomatic Approach to Regularizing Neural Ranking Models](https://mlsanity.com/paper/1904.06808/) (2019)
- [Explainable LLM-driven Multi-dimensional Distillation for E-Commerce Relevance Learning](https://mlsanity.com/paper/2411.13045/) (2024)
- [A Zero-shot Explainable Doctor Ranking Framework with Large Language Models](https://mlsanity.com/paper/2503.02298/) (2025)

## Built on by (most cited first, 11 of 11)

- [Reasoning over Semantic IDs Enhances Generative Recommendation](https://mlsanity.com/paper/2603.23183/) (2026)
- [FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents](https://mlsanity.com/paper/2606.12087/) (2026)
- [PosIR: Position-Aware Heterogeneous Information Retrieval Benchmark](https://mlsanity.com/paper/2601.08363/) (2026)
- [Towards Robust Process Reward Modeling via Noise-aware Learning](https://mlsanity.com/paper/2601.12748/) (2026)
- [SEEK: Skill-Routed Evaluation with Evolvable Knowledge for Industrial Search](https://mlsanity.com/paper/2609.29803/) (2026)
- [Awakening Dormant Users: Generative Recommendation with Counterfactual Functional Role Reasoning](https://mlsanity.com/paper/2602.13134/) (2026)
- [ResRank: Unifying Retrieval and Listwise Reranking via End-to-End Joint Training with Residual Passage Compression](https://mlsanity.com/paper/2604.22180/) (2026)
- [RAG-Match: Retrieval-Augmented Knowledge Injection and Hierarchical Reasoning for Calibrated Semantic Relevance](https://mlsanity.com/paper/2605.25486/) (2026)
- [Tool-Adaptive LLM Reranker](https://mlsanity.com/paper/2607.10555/) (2026)
- [SSR-GRPO: Integrating Supervision and Semantic IDs into Reinforcement Learning for Dense Retrieval in E-commerce](https://mlsanity.com/paper/2608.19595/) (2026)
- [GenMatch: An End-to-End Generative Matching Framework for Micro-View Order-Dispatching in Ride-Hailing](https://mlsanity.com/paper/2608.19751/) (2026)

## Abstract (arXiv)

Ranking relevance is a fundamental task in search engines, aiming to identify the items most relevant to a given user query. Traditional relevance models typically produce scalar scores or directly predict relevance labels, limiting both interpretability and the modeling of complex relevance signals. Inspired by recent advances in Chain-of-Thought (CoT) reasoning for complex tasks, we investigate whether explicit reasoning can enhance both interpretability and performance in relevance modeling. However, existing reasoning-based Generative Relevance Models (GRMs) primarily rely on supervised fine-tuning on large amounts of human-annotated or synthetic CoT data, which often leads to limited generalization. Moreover, domain-agnostic, free-form reasoning tends to be overly generic and insufficiently grounded, limiting its potential to handle the diverse and ambiguous cases prevalent in open-domain search. In this work, we formulate relevance modeling in Xiaohongshu search as a reasoning task and introduce a Reinforcement Learning (RL)-based training framework to enhance the grounded reasoning capabilities of GRMs. Specifically, we incorporate practical business-specific relevance criteria into the multi-step reasoning prompt design and propose Stepwise Advantage Masking (SAM), a lightweight process-supervision strategy which facilitates effective learning of these criteria through improved credit assignment. To enable industrial deployment, we further distill the large-scale RL-tuned model to a lightweight version suitable for real-world search systems. Extensive offline evaluations and online A/B tests demonstrate that our approach consistently delivers significant improvements across key relevance and business metrics, validating its effectiveness, robustness, and practicality for large-scale industrial search systems.