mlsanityData to 14 Oct 2026

Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning

arXiv 2212.03760 ↗First posted 7 Dec 2022cs.IR cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

Recent studies have proposed unified user modeling frameworks that leverage user behavior data from various applications. Many of them benefit from utilizing users' behavior sequences as plain texts, representing rich information in any domain or system without losing generality. Hence, a question arises: Can language modeling for user history corpus help improve recommender systems? While its versatile usability has been widely investigated in many domains, its applications to recommender systems still remain underexplored. We show that language modeling applied directly to task-specific user histories achieves excellent results on diverse recommendation tasks. Also, leveraging additional task-agnostic user histories delivers significant performance benefits. We further demonstrate that our approach can provide promising transfer learning capabilities for a broad spectrum of real-world recommender systems, even on unseen domains and services.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 45 of its 64 references that are in the map. First posted on arXiv 7 Dec 2022.

Builds on

45 of 64 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2020Scaling Laws for Neural Language Modelscited by 6,586
  8. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  11. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,565
  12. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
33 more
  1. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  2. 2018Self-Attentive Sequential Recommendationcited by 1,184
  3. 2017Mixed Precision Trainingcited by 1,156
  4. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,139
  5. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  6. 2021Scaling Vision Transformerscited by 999
  7. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  8. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  9. 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 402
  10. 2021Explaining Neural Scaling Lawscited by 363
  11. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 361
  12. 2022How Do Vision Transformers Work?cited by 307
  13. 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 292
  14. 2021When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentationscited by 260
  15. 2022M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systemscited by 152
  16. 2019Behavior Sequence Transformer for E-commerce Recommendation in Alibabacited by 130
  17. 2022Towards Universal Sequence Representation Learning for Recommender Systemscited by 121
  18. 2019Neural Machine Translation with Byte-Level Subwordscited by 115
  19. 2021Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuningcited by 82
  20. 2022On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Modelcited by 81
  21. 2020A Mathematical Exploration of Why Language Models Help Solve Downstream Taskscited by 72
  22. 2018NSML: Meet the MLaaS platform with a real-world case studycited by 70
  23. 2019An Embarrassingly Simple Approach for Transfer Learning from Pretrained Language Modelscited by 39
  24. 2022Understanding Scaling Laws for Recommendation Modelscited by 38
  25. 2017NSML: A Machine Learning Platform That Enables You to Focus on Your Modelscited by 34
  26. 2021Personalized Transfer of User Preferences for Cross-domain Recommendationcited by 34
  27. 2019DARec: Deep Domain Adaptation for Cross-Domain Recommendation via Transferring Rating Patternscited by 33
  28. 2021UDALM: Unsupervised Domain Adaptation through Language Modelingcited by 33
  29. 2021Scaling Law for Recommendation Models: Towards General-purpose User Representationscited by 26
  30. 2022Downstream Datasets Make Surprisingly Good Pretraining Corporacited by 20
  31. 2020Exploiting Behavioral Consistence for Universal User Representationcited by 14
  32. 2021One4all User Representation for Recommender Systems in E-commercecited by 12
  33. 2020div2vec: Diversity-Emphasized Node Embeddingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 0; 2026: 1.

2 papers.

  1. 2026Affordances Enable Partial World Modeling with LLMscited by 2
  2. 2022Can Current Task-oriented Dialogue Models Automate Real-world Scenarios in the Wild?cited by 1