mlsanityData to 14 Oct 2026

TokenMinds: Pretrained User Tokens and Embeddings for User Understanding in Large Recommender Systems

arXiv 2606.25147 ↗First posted 23 Jun 2026cs.IR cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

User modeling in industrial recommender systems typically produces dense embeddings, which suffer from representational constraints inherent to fixed-dimensional vectors. An emerging alternative for discrete user representation -- using LLMs to generate text-based user tokens -- captures topical co-occurrences rather than deep sequential behavior dynamics and produces outputs that are difficult to ground to item attributes. Meanwhile, Semantic ID (SID) based item tokenization has proven effective for improving generalization in generative recommendation, yet discrete SID-based representations for users remain largely unexplored. We propose TokenMinds, an industrial-scale system that extends the PLUM framework from item retrieval to user modeling, generating both discrete SID-based user tokens and dense user embeddings via an encoder-decoder architecture adapted from pre-trained LLMs. This dual-output design provides the complementary benefits of discrete, semantically grounded user representations while maintaining compatibility with existing downstream models that rely on dense embeddings. Additionally, the shared SID vocabulary naturally extends to cross-scenario modeling: by unifying long-form and short-form video behaviors into a single model, we substantially reduce training and serving costs. We validate TokenMinds through extensive offline experiments and live launches on multiple YouTube surfaces, served on full user traffic (billions of users) via an asynchronous infrastructure that decouples representation generation from downstream scoring. Focusing on ranking as the primary downstream use case, our results confirm the practical viability of SID-based user tokens at industrial scale and demonstrate that tokens and dense embeddings provide complementary value across different production ranking systems.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 22 of its 38 references that are in the map. First posted on arXiv 23 Jun 2026.

Builds on

22 of 38 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  3. 2018Self-Attentive Sequential Recommendationcited by 1,184
  4. 2015Session-based Recommendations with Recurrent Neural Networkscited by 695
  5. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 639
  6. 2018Personalized Top-N Sequential Recommendation via Convolutional Sequence Embeddingcited by 476
  7. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 402
  8. 2025On the Theoretical Limitations of Embedding-Based Retrievalcited by 104
  9. 2023PALR: Personalization Aware LLMs for Recommendationcited by 96
  10. 2022Pathways: Asynchronous Distributed Dataflow for MLcited by 82
  11. 2024IDGenRec: LLM-RecSys Alignment with Textual ID Learningcited by 53
  12. 2025OneRec-Think: In-Text Reasoning for Generative Recommendationcited by 51
10 more
  1. 2025LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenderscited by 47
  2. 2025Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representationscited by 35
  3. 2020Mixed Information Flow for Cross-domain Sequential Recommendationscited by 22
  4. 2024LLM4MSR: An LLM-Enhanced Paradigm for Multi-Scenario Recommendationcited by 19
  5. 2023Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systemscited by 14
  6. 2025GPR: Towards a Generative Pre-trained One-Model Paradigm for Large-Scale Advertising Recommendationcited by 9
  7. 2025DV365: Extremely Long User History Modeling at Instagramcited by 8
  8. 2024MLoRA: Multi-Domain Low-Rank Adaptive Network for CTR Predictioncited by 7
  9. 2021Empowering General-purpose User Representation with Full-life Cycle Behavior Modelingcited by 6
  10. 2022Latent User Intent Modeling for Sequential Recommenderscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.