mlsanityData to 14 Oct 2026

Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation

arXiv 2604.21536 ↗First posted 23 Apr 2026cs.IR cs.AIcited by 0

On the map: Knowledge distillation → Distillation for recommender systems

Abstract (arXiv)

Sequential recommender systems have achieved significant success in modeling temporal user behavior but remain limited in capturing rich user semantics beyond interaction patterns. Large Language Models (LLMs) present opportunities to enhance user understanding with their reasoning capabilities, yet existing integration approaches create prohibitive inference costs in real time. To address these limitations, we present a novel knowledge distillation method that utilizes textual user profile generated by pre-trained LLMs into sequential recommenders without requiring LLM inference at serving time. The resulting approach maintains the inference efficiency of traditional sequential models while requiring neither architectural modifications nor LLM fine-tuning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Distillation for recommender systems, one of 12 directions in Knowledge distillation. Builds on 34 of its 38 references that are in the map. First posted on arXiv 23 Apr 2026.

Builds on

34 of 38 references.

References · 34

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  3. 2023A Survey of Large Language Modelscited by 2,576
  4. 2022Emergent Abilities of Large Language Modelscited by 2,512
  5. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  6. 2018Self-Attentive Sequential Recommendationcited by 1,184
  7. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  8. 2015Image-based Recommendations on Styles and Substitutescited by 706
  9. 2019Predicting Dynamic Embedding Trajectory in Temporal Interaction Networkscited by 448
  10. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 402
  11. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 372
  12. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 309
22 more
  1. 2023Towards Open-World Recommendation with Knowledge Augmentation from Large Language Modelscited by 74
  2. 2023A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systemscited by 73
  3. 2023LlamaRec: Two-Stage Recommendation using Large Language Models for Rankingcited by 53
  4. 2024IDGenRec: LLM-RecSys Alignment with Textual ID Learningcited by 53
  5. 2022TwHIN-BERT: A Socially-Enriched Pre-trained Language Model for Multilingual Tweet Representations at Twittercited by 45
  6. 2023Amazon-M2: A Multilingual Multi-locale Shopping Session Dataset for Recommendation and Text Generationcited by 42
  7. 2020A Critical Study on Data Leakage in Recommender System Offline Evaluationcited by 39
  8. 2023E4SRec: An Elegant Effective Efficient Extensible Solution of Large Language Models for Sequential Recommendationcited by 39
  9. 2021Pre-trained Language Model for Web-scale Retrieval in Baidu Searchcited by 31
  10. 2023Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec?cited by 30
  11. 2024Large Language Models Enhanced Collaborative Filteringcited by 27
  12. 2024Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyondcited by 26
  13. 2024CALRec: Contrastive Alignment of Generative LLMs for Sequential Recommendationcited by 24
  14. 2024LLMEmb: Large Language Model Can Be a Good Embedding Generator for Sequential Recommendationcited by 18
  15. 2024Does It Look Sequential? An Analysis of Datasets for Evaluation of Sequential Recommendationscited by 15
  16. 2023Sparks of Artificial General Recommender (AGR): Early Experiments with ChatGPTcited by 13
  17. 2025Time to Split: Exploring Data Splitting Strategies for Offline Evaluation of Sequential Recommenderscited by 12
  18. 2025GRAM: Generative Recommendation via Semantic-aware Multi-granular Late Fusioncited by 7
  19. 2023TBIN: Modeling Long Textual Behavior Data for CTR Predictioncited by 3
  20. 2025eSASRec: Enhancing Transformer-based Recommendations in a Modular Fashioncited by 3
  21. 2022MTS Kion Implicit Contextualised Sequential Dataset for Movie Recommendationcited by 2
  22. 2024LLM-KT: A Versatile Framework for Knowledge Transfer from Large Language Models to Collaborative Filteringcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Can Small Language Models be Good Reasoners for Sequential Recommendation?
  2. 2024Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Model
  3. 2025Bidirectional Knowledge Distillation for Enhancing Sequential Recommendation with Large Language Models
  4. 2024SLMRec: Distilling Large Language Models into Small for Sequential Recommendation
  5. 2024Unleash LLMs Potential for Sequential Recommendation by Coordinating Dual Dynamic Index Mechanism
  6. 2025Effectiveness of LLMs in Temporal User Profiling for Recommendation
  7. 2024LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation
  8. 2026LLM-Based User Personas for Recommendations at Scale