# Generating Query-Relevant Document Summaries via Reinforcement Learning

arXiv: https://arxiv.org/abs/2508.08404 · HTML: https://mlsanity.com/paper/2508.08404/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 11 Aug 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [BooookScore: A systematic exploration of book-length summarization in the era of LLMs](https://mlsanity.com/paper/2310.00785/) (2023)
- [An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT](https://mlsanity.com/paper/2304.08448/) (2023)
- [SummIt: Iterative Text Summarization via ChatGPT](https://mlsanity.com/paper/2305.14835/) (2023)
- [Text Summarization Using Large Language Models: A Comparative Study of MPT-7b-instruct, Falcon-7b-instruct, and OpenAI Chat-GPT Models](https://mlsanity.com/paper/2310.10449/) (2023)
- [Gradient-Based Language Model Red Teaming](https://mlsanity.com/paper/2401.16656/) (2024)
- [Prompt Chaining or Stepwise Prompt? Refinement in Text Summarization](https://mlsanity.com/paper/2406.00507/) (2024)
- [InstructPTS: Instruction-Tuning LLMs for Product Title Summarization](https://mlsanity.com/paper/2310.16361/) (2023)
- [Enhancing Relevance of Embedding-based Retrieval at Walmart](https://mlsanity.com/paper/2408.04884/) (2024)
- [Explainable LLM-driven Multi-dimensional Distillation for E-Commerce Relevance Learning](https://mlsanity.com/paper/2411.13045/) (2024)
- [Doc2Token: Bridging Vocabulary Gap by Predicting Missing Tokens for E-commerce Search](https://mlsanity.com/paper/2406.19647/) (2024)

## Built on by (most cited first, 1 of 1)

- [Think Before You Retrieve: Learning Test-Time Adaptive Search with Small Language Models](https://mlsanity.com/paper/2511.07581/) (2025)

## Abstract (arXiv)

E-commerce search engines often rely solely on product titles as input for ranking models with latency constraints. However, this approach can result in suboptimal relevance predictions, as product titles often lack sufficient detail to capture query intent. While product descriptions provide richer information, their verbosity and length make them unsuitable for real-time ranking, particularly for computationally expensive architectures like cross-encoder ranking models. To address this challenge, we propose ReLSum, a novel reinforcement learning framework designed to generate concise, query-relevant summaries of product descriptions optimized for search relevance. ReLSum leverages relevance scores as rewards to align the objectives of summarization and ranking, effectively overcoming limitations of prior methods, such as misaligned learning targets. The framework employs a trainable large language model (LLM) to produce summaries, which are then used as input for a cross-encoder ranking model. Experimental results demonstrate significant improvements in offline metrics, including recall and NDCG, as well as online user engagement metrics. ReLSum provides a scalable and efficient solution for enhancing search relevance in large-scale e-commerce systems.