xGR: Efficient Generative Recommendation Serving at Scale
On the map: Efficient LLM inference and serving → KV cache optimization
Abstract (arXiv)
Recommendation system delivers substantial economic benefits by providing personalized predictions. Generative recommendation (GR) integrates LLMs to enhance the understanding of long user-item sequences. Despite employing attention-based architectures, GR's workload differs markedly from that of LLM serving. GR typically processes long prompt while producing short, fixed-length outputs, yet the computational cost of each decode phase is especially high due to the large beam width. Furthermore, since the beam search involves a vast item space, the sorting overhead becomes particularly time-consuming. We propose xGR, a GR-oriented serving system that meets strict low-latency requirements under high-concurrency scenarios. First, xGR unifies the processing of prefill and decode phases through staged computation and separated KV cache. Second, xGR enables early sorting termination and mask-based item filtering with data structure reuse. Third, xGR reconstructs the overall pipeline to exploit multi-level overlap and multi-stream parallelism. The experiments on real-world datasets demonstrate that xGR achieves at least 2.89x throughput compared to the state-of-the-art baseline under strict latency constraints.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 39 of its 55 references that are in the map. First posted on arXiv 12 Dec 2025.
Builds on
39 of 55 references.
Closest ancestors · 3
Same direction, newest first
- 2024DiffKV: Differentiated Memory Management for Large Language Models with Parallel KV Compaction4 Dec
- 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion26 May
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
Other references · 36
- 2017Attention Is All You Needcited by 59,932
- 2023GPT-4 Technical Reportcited by 18,132
- 2025Qwen3 Technical Reportcited by 8,195
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
- 2016Wide & Deep Learning for Recommender Systemscited by 745
- 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 673
- 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 595
- 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 536
- 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
24 more
- 2024Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoderscited by 213
- 2023Self-Evaluation Guided Beam Search for Reasoningcited by 195
- 2023Recommender Systems with Generative Retrievalcited by 191
- 2024Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendationscited by 184
- 2025OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignmentcited by 167
- 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 141
- 2020If beam search is the answer, what was the question?cited by 114
- 2019Robust Navigation with Language Pretraining and Stochastic Samplingcited by 77
- 2024Hydragen: High-Throughput LLM Inference with Shared Prefixescited by 71
- 2024A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)cited by 58
- 2021Machine Translation Decoding beyond Beam Searchcited by 55
- 2025MTGR: Industrial-Scale Generative Recommendation Framework in Meituancited by 47
- 2022Monolith: Real Time Recommendation System With Collisionless Embedding Tablecited by 30
- 2025PrefillOnly: An Inference Engine for Prefill-only Workloads in Large Language Model Applicationscited by 21
- 2025MiniOneRec: An Open-Source Framework for Scaling Generative Recommendationcited by 16
- 2024GMLake: Efficient and Transparent GPU Memory Defragmentation for Large-scale DNN Training with Virtual Memory Stitchingcited by 13
- 2024RelayAttention for Efficient Large Language Model Serving with Long System Promptscited by 13
- 2024Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusterscited by 13
- 2025IC-Cache: Efficient Large Language Model Serving via In-context Cachingcited by 13
- 2024Efficient Inference for Large Language Model-based Generative Recommendationcited by 11
- 2025Scaling Transformers for Discriminative Recommendation via Generative Pretrainingcited by 10
- 2022Hercules: Heterogeneity-Aware Inference Serving for At-Scale Personalized Recommendationcited by 9
- 2022A GPU-specialized Inference Parameter Server for Large-Scale Deep Recommendation Modelscited by 9
- 2025xLLM Technical Reportcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026RelayGR: Scaling Long-Sequence Generative Recommendation via Cross-Stage Relay-Race Inference
- 2026GEMs: Breaking the Long-Sequence Barrier in Generative Recommendation with a Multi-Stream Decoder
- 2025NEZHA: A Zero-sacrifice and Hyperspeed Decoding Architecture for Generative Recommendations
- 2025GR-LLMs: Recent Advances in Generative Recommendation Based on Large Language Models
- 2026Learning to Reflect and Correct: Towards Better Decoding Trajectories for Large-Scale Generative Recommendation
- 2026GenRec: A Preference-Oriented Generative Framework for Large-Scale Recommendation
- 2025Generative Large Recommendation Models: Emerging Trends in LLMs for Recommendation
- 2025GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networks