mlsanityData to 14 Oct 2026

MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches

arXiv 2604.22881 ↗First posted 24 Apr 2026cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Generative recommendation (GR) offers superior modeling capabilities but suffers from prohibitive inference costs due to the repeated encoding of long user histories. While cross-request Key-Value (KV) cache reuse presents a significant optimization opportunity, the massive scale of individual user states creates a storage explosion that far exceeds physical GPU limits. We propose MTServe, a hierarchical cache management system that virtualizes GPU memory by leveraging host RAM as a scalable backup store. To bridge the I/O gap between tiers, MTServe introduces a suite of system-level optimizations, including a hybrid storage layout, an asynchronous data transfer pipeline, and a locality-driven replacement policy. On both public and production datasets, MTServe delivers up to 3.1* speedup while maintaining near-perfect hit ratios (>98.5%).

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 21 of its 37 references that are in the map. First posted on arXiv 24 Apr 2026.

Builds on

21 of 37 references.

References · 21

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2018Self-Attentive Sequential Recommendationcited by 1,184
  3. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  4. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  5. 2016Wide & Deep Learning for Recommender Systemscited by 747
  6. 2017DeepFM: A Factorization-Machine based Neural Network for CTR Predictioncited by 676
  7. 2017Deep & Cross Network for Ad Click Predictionscited by 393
  8. 2018Deep Interest Evolution Network for Click-Through Rate Predictioncited by 338
  9. 2025OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignmentcited by 173
  10. 2018Billion-scale Commodity Embedding for E-commerce Recommendation in Alibabacited by 105
  11. 2024HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modelingcited by 63
  12. 2025MTGR: Industrial-Scale Generative Recommendation Framework in Meituancited by 49
9 more
  1. 2025OneTrans: Unified Feature Interaction and Sequence Modeling with One Transformer in Industrial Recommendercited by 35
  2. 2024Generative Recommender with End-to-End Learnable Item Tokenizationcited by 34
  3. 2024CALRec: Contrastive Alignment of Generative LLMs for Sequential Recommendationcited by 24
  4. 2025OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking Systemcited by 18
  5. 2024Inductive Generative Recommendation via Retrieval-based Speculationcited by 15
  6. 2024Bridging Search and Recommendation in Generative Retrieval: Does One Task Help the Other?cited by 7
  7. 2024Preference Discerning with LLM-Enhanced Generative Retrievalcited by 6
  8. 2024ULMRec: User-centric Large Language Model for Sequential Recommendationcited by 4
  9. 2026Token-level Collaborative Alignment for LLM-based Generative Recommendationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Enabling High-Bandwidth Flash for Generative Recommendation Serving with Write-Aware KV Cache Policycited by 0
  2. 2026Self-Evolving Memory for Generative Recommendationcited by 0