# PLUM: Adapting Pre-trained Language Models for Industrial-scale Generative Recommendations

arXiv: https://arxiv.org/abs/2510.07784 · HTML: https://mlsanity.com/paper/2510.07784/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 44 references that are in the map.
- First posted on arXiv 9 Oct 2025.

## Builds on (most cited first, 17 of 17)

- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [Transformer Memory as a Differentiable Search Index](https://mlsanity.com/paper/2202.06991/) (2022)
- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [Top-K Off-Policy Correction for a REINFORCE Recommender System](https://mlsanity.com/paper/1812.02353/) (2018)
- [Behavior Sequence Transformer for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1905.06874/) (2019)
- [On the Theoretical Limitations of Embedding-Based Retrieval](https://mlsanity.com/paper/2508.21038/) (2025)
- [Better Generalization with Semantic IDs: A Case Study in Ranking for Recommendations](https://mlsanity.com/paper/2306.08121/) (2023)
- [Monolith: Real Time Recommendation System With Collisionless Embedding Table](https://mlsanity.com/paper/2209.07663/) (2022)
- [Enhancing Embedding Representation Stability in Recommendation Systems with Semantic ID](https://mlsanity.com/paper/2504.02137/) (2025)
- [Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems](https://mlsanity.com/paper/2305.12102/) (2023)
- [Generative Retrieval and Alignment Model: A New Paradigm for E-commerce Retrieval](https://mlsanity.com/paper/2504.01403/) (2025)
- [360Brew: A Decoder-only Foundation Model for Personalized Ranking and Recommendation](https://mlsanity.com/paper/2501.16450/) (2025)
- [PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform](https://mlsanity.com/paper/2507.12704/) (2025)
- [BBQRec: Behavior-Bind Quantization for Multi-Modal Sequential Recommendation](https://mlsanity.com/paper/2504.06636/) (2025)

## Built on by (most cited first, 30 of 30)

- [Reasoning over Semantic IDs Enhances Generative Recommendation](https://mlsanity.com/paper/2603.23183/) (2026)
- [GR2: Generative Reasoning Re-ranker](https://mlsanity.com/paper/2602.07774/) (2026)
- [Ad Insertion in LLM-Generated Responses](https://mlsanity.com/paper/2601.19435/) (2026)
- [Generative Recommendation for Large-Scale Advertising](https://mlsanity.com/paper/2602.22732/) (2026)
- [S2GR: Stepwise Semantic-Guided Reasoning in Latent Space for Generative Recommendation](https://mlsanity.com/paper/2601.18664/) (2026)
- [Vectorizing the Trie: Efficient Constrained Decoding for LLM-based Generative Retrieval on Accelerators](https://mlsanity.com/paper/2602.22647/) (2026)
- [Variable-Length Semantic IDs for Recommender Systems](https://mlsanity.com/paper/2602.16375/) (2026)
- [AgenticTagger: Structured Item Representation for Recommendation with LLM Agents](https://mlsanity.com/paper/2602.05945/) (2026)
- [A Unified Language Model for Large Scale Search, Recommendation, and Reasoning](https://mlsanity.com/paper/2603.17533/) (2026)
- [LocalSUG: City-Preference-Enhanced LLM for Query Suggestion in Local-Life Services](https://mlsanity.com/paper/2603.04946/) (2026)
- [From Relevance to Authority: Authority-aware Generative Retrieval in Web Search Engines](https://mlsanity.com/paper/2604.13468/) (2026)
- [SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendation](https://mlsanity.com/paper/2605.18920/) (2026)
- [Recommendation Retrievers Need Verifiers: Universal Generative Reranking for Sequential Recommendations](https://mlsanity.com/paper/2609.12270/) (2026)
- [OneLA: Scaling Linear-Attention Decoding to Large Beams in Generative Recommendation](https://mlsanity.com/paper/2609.12399/) (2026)
- [Principled Synthetic Data Enables the First Scaling Laws for LLMs in Recommendation](https://mlsanity.com/paper/2602.07298/) (2026)
- [LLM Biases](https://mlsanity.com/paper/2604.26960/) (2026)
- [UxSID: Semantic-Aware User Interests Modeling for Ultra-Long Sequence](https://mlsanity.com/paper/2605.09040/) (2026)
- [On the Memorization Behavior of LLMs in Generative Recommendation: Observations, Implications, and Training Strategies](https://mlsanity.com/paper/2606.17276/) (2026)
- [GR2 Technical Report](https://mlsanity.com/paper/2606.31984/) (2026)
- [Diffusion-GR2: Diffusion Generative Reasoning Re-ranker](https://mlsanity.com/paper/2607.01170/) (2026)
- [An LLM-powered Agentic Recommendation System for Connected TV Content Discovery](https://mlsanity.com/paper/2607.09988/) (2026)
- [Tokens are All You Need: Dual-purpose Semantic IDs for Achieving LLM-Level I/O Efficiency in recommendation systems](https://mlsanity.com/paper/2607.24865/) (2026)
- [OneShot: Index-in-Ranking with Neural Scoring for Large-Scale Retrieval](https://mlsanity.com/paper/2607.27475/) (2026)
- [Interpretable Representation via LLM-Driven Generative Disentanglement for Local-Life Service Recommendation](https://mlsanity.com/paper/2607.27944/) (2026)
- [SA-RSQ: A Versatile Sparse Representation Framework for Multi-modal Recommender Systems](https://mlsanity.com/paper/2608.22979/) (2026)
- [Self-Evolving Memory for Generative Recommendation](https://mlsanity.com/paper/2609.15598/) (2026)
- [Guiding the coarse levels of semantic IDs makes the fine levels learnable](https://mlsanity.com/paper/2609.22227/) (2026)
- [Bootstrapping Conversational Recommendation Agents At Spotify: Synthetic Data Generation and Self-Improvement Loops](https://mlsanity.com/paper/2609.30297/) (2026)
- [KuaFu: Compressing Long User Behavior into Understanding at Billion Scale](https://mlsanity.com/paper/2609.31045/) (2026)
- [Personalized LLM Recommendation with Semantic IDs: Autoregressive Beam Search vs. Multi-Token Prediction](https://mlsanity.com/paper/2610.15864/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) pose a new paradigm of modeling and computation for information tasks. Recommendation systems are a critical application domain poised to benefit significantly from the sequence modeling capabilities and world knowledge inherent in these large models. In this paper, we introduce PLUM, a framework designed to adapt pre-trained LLMs for industry-scale recommendation tasks. PLUM consists of item tokenization using Semantic IDs, continued pre-training (CPT) on domain-specific data, and task-specific fine-tuning for recommendation objectives. For fine-tuning, we focus particularly on generative retrieval, where the model is directly trained to generate Semantic IDs of recommended items based on user context. We conduct comprehensive experiments on large-scale internal video recommendation datasets. Our results demonstrate that PLUM achieves substantial improvements for retrieval compared to a heavily-optimized production model built with large embedding tables. We also present a scaling study for the model's retrieval performance, our learnings about CPT, a few enhancements to Semantic IDs, along with an overview of the training and inference methods that enable launching this framework to billions of users in YouTube.