# LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation

arXiv: https://arxiv.org/abs/2405.20646 · HTML: https://mlsanity.com/paper/2405.20646/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 66 references that are in the map.
- First posted on arXiv 31 May 2024.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [A Survey on Large Language Models for Recommendation](https://mlsanity.com/paper/2305.19860/) (2023)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [Improving Text Embeddings with Large Language Models](https://mlsanity.com/paper/2401.00368/) (2023)
- [Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System](https://mlsanity.com/paper/2303.14524/) (2023)
- [Recommender Systems in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2307.02046/) (2023)
- [Representation Learning with Large Language Models for Recommendation](https://mlsanity.com/paper/2310.15950/) (2023)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [Pre-training of Graph Augmented Transformers for Medication Recommendation](https://mlsanity.com/paper/1906.00346/) (2019)
- [Sequential Recommender Systems: Challenges, Progress and Prospects](https://mlsanity.com/paper/2001.04830/) (2019)
- [When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications](https://mlsanity.com/paper/2310.18339/) (2023)
- [Filter-enhanced MLP is All You Need for Sequential Recommendation](https://mlsanity.com/paper/2202.13556/) (2022)
- [Sequential Recommendation with Graph Neural Networks](https://mlsanity.com/paper/2106.14226/) (2021)
- [Leveraging Large Language Models for Sequential Recommendation](https://mlsanity.com/paper/2309.09261/) (2023)
- [Large Language Models for Generative Recommendation: A Survey and Visionary Discussions](https://mlsanity.com/paper/2309.01157/) (2023)
- [Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer](https://mlsanity.com/paper/2105.00522/) (2021)
- [Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations](https://mlsanity.com/paper/2307.05722/) (2023)
- [E4SRec: An Elegant Effective Efficient Extensible Solution of Large Language Models for Sequential Recommendation](https://mlsanity.com/paper/2312.02443/) (2023)

## Built on by (most cited first, 30 of 47)

- [A Survey of Personalization: From RAG to Agent](https://mlsanity.com/paper/2504.10147/) (2025)
- [Multi-Pair Temporal Sentence Grounding via Multi-Thread Knowledge Transfer Network](https://mlsanity.com/paper/2412.15678/) (2024)
- [Bridge the Domains: Large Language Models Enhanced Cross-domain Sequential Recommendation](https://mlsanity.com/paper/2504.18383/) (2025)
- [LLM-RecG: A Semantic Bias-Aware Framework for Zero-Shot Sequential Recommendation](https://mlsanity.com/paper/2501.19232/) (2025)
- [Semantic Retrieval Augmented Contrastive Learning for Sequential Recommendation](https://mlsanity.com/paper/2503.04162/) (2025)
- [Training-free LLM Merging for Multi-task Learning](https://mlsanity.com/paper/2506.12379/) (2025)
- [Inference Computation Scaling for Feature Augmentation in Recommendation Systems](https://mlsanity.com/paper/2502.16040/) (2025)
- [Pre-train, Align, and Disentangle: Empowering Sequential Recommendation with Large Language Models](https://mlsanity.com/paper/2412.04107/) (2024)
- [Field Matters: A Lightweight LLM-enhanced Method for CTR Prediction](https://mlsanity.com/paper/2505.14057/) (2025)
- [Bidirectional Knowledge Distillation for Enhancing Sequential Recommendation with Large Language Models](https://mlsanity.com/paper/2505.18120/) (2025)
- [FlowRec: Prior-Informed Flow Matching for Efficient Sequential Recommendation Generation](https://mlsanity.com/paper/2508.17618/) (2025)
- [Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning](https://mlsanity.com/paper/2509.18942/) (2025)
- [TAPO: Task-Referenced Adaptation for Prompt Optimization](https://mlsanity.com/paper/2501.06689/) (2025)
- [GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networks](https://mlsanity.com/paper/2506.16114/) (2025)
- [CoVE: Compressed Vocabulary Expansion Makes Better LLM-based Recommender Systems](https://mlsanity.com/paper/2506.19993/) (2025)
- [Tuning Language Models for Robust Prediction of Diverse User Behaviors](https://mlsanity.com/paper/2505.17682/) (2025)
- [Model Merging for Knowledge Editing](https://mlsanity.com/paper/2506.12384/) (2025)
- [Not Just What, But When: Integrating Irregular Intervals to LLM for Sequential Recommendation](https://mlsanity.com/paper/2507.23209/) (2025)
- [Empowering Large Language Model for Sequential Recommendation via Multimodal Embeddings and Semantic IDs](https://mlsanity.com/paper/2509.02017/) (2025)
- [Enhancing Time Awareness in Generative Recommendation](https://mlsanity.com/paper/2509.13957/) (2025)
- [Distilling Transitional Pattern to Large Language Models for Multimodal Session-based Recommendation](https://mlsanity.com/paper/2504.10538/) (2025)
- [A Multi-Expert Structural-Semantic Hybrid Framework for Unveiling Historical Patterns in Temporal Knowledge Graphs](https://mlsanity.com/paper/2506.14235/) (2025)
- [M-LLM3REC: A Motivation-Aware User-Item Interaction Framework for Enhancing Recommendation Accuracy with LLMs](https://mlsanity.com/paper/2508.15262/) (2025)
- [ChoirRec: Semantic User Grouping via LLMs for Conversion Rate Prediction of Low-Activity Users](https://mlsanity.com/paper/2510.09393/) (2025)
- [LLM-EDT: Large Language Model Enhanced Cross-domain Sequential Recommendation with Dual-phase Training](https://mlsanity.com/paper/2511.19931/) (2025)
- [MMSRARec: Summarization and Retrieval Augumented Sequential Recommendation Based on Multimodal Large Language Model](https://mlsanity.com/paper/2512.20916/) (2025)
- [Generative Archetype-Grounded Item Representations for Sequential Recommendation](https://mlsanity.com/paper/2606.11023/) (2026)
- [ShiJianBench: From Dialogue to Decision for Long-Horizon Evaluation of Investment Advisors](https://mlsanity.com/paper/2608.01204/) (2026)
- [EGRA:Toward Enhanced Behavior Graphs and Representation Alignment for Multimodal Recommendation](https://mlsanity.com/paper/2508.16170/) (2025)
- [SemSR: Semantics aware robust Session-based Recommendations](https://mlsanity.com/paper/2508.20587/) (2025)

## Abstract (arXiv)

Sequential recommender systems (SRS) aim to predict users' subsequent choices based on their historical interactions and have found applications in diverse fields such as e-commerce and social media. However, in real-world systems, most users interact with only a handful of items, while the majority of items are seldom consumed. These two issues, known as the long-tail user and long-tail item challenges, often pose difficulties for existing SRS. These challenges can adversely affect user experience and seller benefits, making them crucial to address. Though a few works have addressed the challenges, they still struggle with the seesaw or noisy issues due to the intrinsic scarcity of interactions. The advancements in large language models (LLMs) present a promising solution to these problems from a semantic perspective. As one of the pioneers in this field, we propose the Large Language Models Enhancement framework for Sequential Recommendation (LLM-ESR). This framework utilizes semantic embeddings derived from LLMs to enhance SRS without adding extra inference load from LLMs. To address the long-tail item challenge, we design a dual-view modeling framework that combines semantics from LLMs and collaborative signals from conventional SRS. For the long-tail user challenge, we propose a retrieval augmented self-distillation method to enhance user preference representation using more informative interactions from similar users. To verify the effectiveness and versatility of our proposed enhancement framework, we conduct extensive experiments on three real-world datasets using three popular SRS models. The results show that our method surpasses existing baselines consistently, and benefits long-tail users and items especially. The implementation code is available at https://github.com/Applied-Machine-Learning-Lab/LLM-ESR.