# LLM2Rec: Large Language Models Are Powerful Embedding Models for Sequential Recommendation

arXiv: https://arxiv.org/abs/2506.21579 · HTML: https://mlsanity.com/paper/2506.21579/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 62 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 30 of 39)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Self-supervised Graph Learning for Recommendation](https://mlsanity.com/paper/2010.10783/) (2020)
- [Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders](https://mlsanity.com/paper/2403.03952/) (2024)
- [SGPT: GPT Sentence Embeddings for Semantic Search](https://mlsanity.com/paper/2202.08904/) (2022)
- [LLMRec: Large Language Models with Graph Augmentation for Recommendation](https://mlsanity.com/paper/2311.00423/) (2023)
- [Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach](https://mlsanity.com/paper/2305.07001/) (2023)
- [Recurrent Neural Networks with Top-k Gains for Session-based Recommendations](https://mlsanity.com/paper/1706.03847/) (2017)
- [Representation Learning with Large Language Models for Recommendation](https://mlsanity.com/paper/2310.15950/) (2023)
- [Text Is All You Need: Learning Language Representations for Sequential Recommendation](https://mlsanity.com/paper/2305.13731/) (2023)
- [Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited](https://mlsanity.com/paper/2303.13835/) (2023)
- [Towards Universal Sequence Representation Learning for Recommender Systems](https://mlsanity.com/paper/2206.05941/) (2022)
- [Contrastive Learning for Cold-Start Recommendation](https://mlsanity.com/paper/2107.05315/) (2021)
- [Making Text Embedders Few-Shot Learners](https://mlsanity.com/paper/2409.15700/) (2024)

## Built on by (most cited first, 19 of 19)

- [Latent Inter-User Difference Modeling for LLM Personalization](https://mlsanity.com/paper/2507.20849/) (2025)
- [Understanding Generative Recommendation with Semantic IDs from a Model-scaling View](https://mlsanity.com/paper/2509.25522/) (2025)
- [DiffGRM: Diffusion-based Generative Recommendation Model](https://mlsanity.com/paper/2510.21805/) (2025)
- [Enhancing Foundation Models in Transaction Understanding with LLM-based Sentence Embeddings](https://mlsanity.com/paper/2601.05271/) (2025)
- [Preference-Aware Rubric Learning for Personalized Evaluation](https://mlsanity.com/paper/2605.31545/) (2026)
- [Instruction-aware User Embedding via Synergistic Language and Representation Modeling](https://mlsanity.com/paper/2510.11016/) (2025)
- [REVISION:Reflective Intent Mining and Online Reasoning Auxiliary for E-commerce Visual Search System Optimization](https://mlsanity.com/paper/2510.22739/) (2025)
- [ORBIT -- Open Recommendation Benchmark for Reproducible Research with Hidden Tests](https://mlsanity.com/paper/2510.26095/) (2025)
- [MMGRid: Navigating Temporal-aware and Cross-domain Generative Recommendation via Model Merging](https://mlsanity.com/paper/2601.15930/) (2026)
- [High Fidelity Textual User Representation over Heterogeneous Sources via Reinforcement Learning](https://mlsanity.com/paper/2602.07333/) (2026)
- [Query as Anchor: Scenario-Adaptive User Representation via Large Language Model](https://mlsanity.com/paper/2602.14492/) (2026)
- [Beyond Semantic Understanding: Preserving Collaborative Frequency Components in LLM-based Recommendation](https://mlsanity.com/paper/2508.10312/) (2025)
- [Sensory-Aware Sequential Recommendation via Review-Distilled Representations](https://mlsanity.com/paper/2603.02709/) (2026)
- [VLM2Rec: Resolving Modality Collapse in Vision-Language Model Embedders for Multimodal Sequential Recommendation](https://mlsanity.com/paper/2603.17450/) (2026)
- [Bridging Textual Profiles and Latent User Embeddings for Personalization](https://mlsanity.com/paper/2605.06981/) (2026)
- [LLM-Assisted Reranking to Operationalize Nuanced Objectives in Recommender Systems](https://mlsanity.com/paper/2606.02883/) (2026)
- [CALMRec: Causally Aligned Language Memory for Long-Horizon Recommendation](https://mlsanity.com/paper/2607.23647/) (2026)
- [Behavior2Trip: Towards Personalized Travel Planning via User Behavior Trajectory](https://mlsanity.com/paper/2608.26807/) (2026)
- [Explainable Recommendations at Scale: LLM Rationales for YouTube Music Artist Discovery](https://mlsanity.com/paper/2609.23877/) (2026)

## Abstract (arXiv)

Sequential recommendation aims to predict users' future interactions by modeling collaborative filtering (CF) signals from historical behaviors of similar users or items. Traditional sequential recommenders predominantly rely on ID-based embeddings, which capture CF signals through high-order co-occurrence patterns. However, these embeddings depend solely on past interactions, lacking transferable knowledge to generalize to unseen domains. Recent advances in large language models (LLMs) have motivated text-based recommendation approaches that derive item representations from textual descriptions. While these methods enhance generalization, they fail to encode CF signals-i.e., latent item correlations and preference patterns-crucial for effective recommendation. We argue that an ideal embedding model should seamlessly integrate CF signals with rich semantic representations to improve both in-domain and out-of-domain recommendation performance. To this end, we propose LLM2Rec, a novel embedding model tailored for sequential recommendation, integrating the rich semantic understanding of LLMs with CF awareness. Our approach follows a two-stage training framework: (1) Collaborative Supervised Fine-tuning, which adapts LLMs to infer item relationships based on historical interactions, and (2) Item-level Embedding Modeling, which refines these specialized LLMs into structured item embedding models that encode both semantic and collaborative information. Extensive experiments on real-world datasets demonstrate that LLM2Rec effectively improves recommendation quality across both in-domain and out-of-domain settings. Our findings highlight the potential of leveraging LLMs to build more robust, generalizable embedding models for sequential recommendation. Our codes are available at https://github.com/HappyPointer/LLM2Rec.