mlsanityData to 7 Oct 2026

The Elephant in the Room: Rethinking the Usage of Pre-trained Language Model in Sequential Recommendation

arXiv 2404.08796 ↗First posted 12 Apr 2024cs.IRcited by 0

On the map: LLM reasoning → LLM reasoning for recommendation

Abstract (arXiv)

Sequential recommendation (SR) has seen significant advancements with the help of Pre-trained Language Models (PLMs). Some PLM-based SR models directly use PLM to encode user historical behavior's text sequences to learn user representations, while there is seldom an in-depth exploration of the capability and suitability of PLM in behavior sequence modeling. In this work, we first conduct extensive model analyses between PLMs and PLM-based SR models, discovering great underutilization and parameter redundancy of PLMs in behavior sequence modeling. Inspired by this, we explore different lightweight usages of PLMs in SR, aiming to maximally stimulate the ability of PLMs for SR while satisfying the efficiency and usability demands of practical systems. We discover that adopting behavior-tuned PLMs for item initializations of conventional ID-based SR models is the most economical framework of PLM-based SR, which would not bring in any additional inference cost but could achieve a dramatic performance boost compared with the original version. Extensive experiments on five datasets show that our simple and universal framework leads to significant improvement compared to classical SR and SOTA PLM-based SR models without additional inference costs. Our code can be found in https://github.com/777pomingzi/Rethinking-PLM-in-RS.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in LLM reasoning for recommendation, one of 17 directions in LLM reasoning. Builds on 18 of its 50 references that are in the map. First posted on arXiv 12 Apr 2024.

Builds on

18 of 50 references.

References · 18

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  3. 2020Longformer: The Long-Document Transformercited by 3,362
  4. 2015Session-based Recommendations with Recurrent Neural Networkscited by 685
  5. 2023TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendationcited by 336
  6. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
  7. 2023Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender Systemcited by 241
  8. 2023Is ChatGPT a Good Recommender? A Preliminary Studycited by 192
  9. 2023LLMRec: Large Language Models with Graph Augmentation for Recommendationcited by 171
  10. 2023Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approachcited by 168
  11. 2023Representation Learning with Large Language Models for Recommendationcited by 155
  12. 2022M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systemscited by 152
6 more
  1. 2023Text Is All You Need: Learning Language Representations for Sequential Recommendationcited by 149
  2. 2023Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisitedcited by 124
  3. 2023Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Predictioncited by 113
  4. 2023A Bi-Step Grounding Paradigm for Large Language Models in Recommendation Systemscited by 73
  5. 2023Zero-Shot Next-Item Recommendation using Large Pretrained Language Modelscited by 53
  6. 2023Thoroughly Modeling Multi-domain Pre-trained Recommendation as Languagecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation
  2. 2024Can Small Language Models be Good Reasoners for Sequential Recommendation?
  3. 2024LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation
  4. 2024Rethinking Large Language Model Architectures for Sequential Recommendations
  5. 2024Customizing Language Models with Instance-wise LoRA for Sequential Recommendation
  6. 2025What Makes LLMs Effective Sequential Recommenders? A Study on Preference Intensity and Temporal Context
  7. 2025An item is worth one token in Multimodal Large Language Models-based Sequential Recommendation
  8. 2024LLM-based Bi-level Multi-interest Learning Framework for Sequential Recommendation