# Text Is All You Need: Learning Language Representations for Sequential Recommendation

arXiv: https://arxiv.org/abs/2305.13731 · HTML: https://mlsanity.com/paper/2305.13731/

## Key facts

- Cited by 149 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2023.

## Built on by (most cited first, 30 of 149)

- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [Representation Learning with Large Language Models for Recommendation](https://mlsanity.com/paper/2310.15950/) (2023)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation](https://mlsanity.com/paper/2308.11131/) (2023)
- [Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Efforts](https://mlsanity.com/paper/2406.10471/) (2024)
- [HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling](https://mlsanity.com/paper/2409.12740/) (2024)
- [LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking](https://mlsanity.com/paper/2311.02089/) (2023)
- [IDGenRec: LLM-RecSys Alignment with Textual ID Learning](https://mlsanity.com/paper/2403.19021/) (2024)
- [Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation](https://mlsanity.com/paper/2503.22675/) (2025)
- [Large Language Models meet Collaborative Filtering: An Efficient All-round LLM-based Recommender System](https://mlsanity.com/paper/2404.11343/) (2024)
- [eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data](https://mlsanity.com/paper/2402.08831/) (2024)
- [A Survey on Personalized Alignment -- The Missing Piece for Large Language Models in Real-World Applications](https://mlsanity.com/paper/2503.17003/) (2025)
- [Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations](https://mlsanity.com/paper/2503.02453/) (2025)
- [Customizing Language Models with Instance-wise LoRA for Sequential Recommendation](https://mlsanity.com/paper/2408.10159/) (2024)
- [ClickPrompt: CTR Models are Strong Prompt Generators for Adapting Language Models to CTR Prediction](https://mlsanity.com/paper/2310.09234/) (2023)
- [DRDT: Dynamic Reflection with Divergent Thinking for LLM-based Sequential Recommendation](https://mlsanity.com/paper/2312.11336/) (2023)
- [Rethinking Large Language Model Architectures for Sequential Recommendations](https://mlsanity.com/paper/2402.09543/) (2024)
- [Can Small Language Models be Good Reasoners for Sequential Recommendation?](https://mlsanity.com/paper/2403.04260/) (2024)
- [Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyond](https://mlsanity.com/paper/2403.10667/) (2024)
- [CALRec: Contrastive Alignment of Generative LLMs for Sequential Recommendation](https://mlsanity.com/paper/2405.02429/) (2024)
- [Breaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors](https://mlsanity.com/paper/2403.19347/) (2024)
- [LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application](https://mlsanity.com/paper/2405.03988/) (2024)
- [Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models](https://mlsanity.com/paper/2410.20745/) (2024)
- [Enhancing Recommendation Diversity by Re-ranking with Large Language Models](https://mlsanity.com/paper/2401.11506/) (2024)
- [Stealthy Attack on Large Language Model based Recommendation](https://mlsanity.com/paper/2402.14836/) (2024)
- [LLM2Rec: Large Language Models Are Powerful Embedding Models for Sequential Recommendation](https://mlsanity.com/paper/2506.21579/) (2025)
- [Unifying Generative and Dense Retrieval for Sequential Recommendation](https://mlsanity.com/paper/2411.18814/) (2024)
- [Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation](https://mlsanity.com/paper/2408.03533/) (2024)
- [A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigms](https://mlsanity.com/paper/2504.16420/) (2025)
- [Federated Recommendation via Hybrid Retrieval Augmented Generation](https://mlsanity.com/paper/2403.04256/) (2024)

## Abstract (arXiv)

Sequential recommendation aims to model dynamic user behavior from historical interactions. Existing methods rely on either explicit item IDs or general textual features for sequence modeling to understand user preferences. While promising, these approaches still struggle to model cold-start items or transfer knowledge to new datasets. In this paper, we propose to model user preferences and item features as language representations that can be generalized to new items and datasets. To this end, we present a novel framework, named Recformer, which effectively learns language representations for sequential recommendation. Specifically, we propose to formulate an item as a "sentence" (word sequence) by flattening item key-value attributes described by text so that an item sequence for a user becomes a sequence of sentences. For recommendation, Recformer is trained to understand the "sentence" sequence and retrieve the next "sentence". To encode item sequences, we design a bi-directional Transformer similar to the model Longformer but with different embedding layers for sequential recommendation. For effective representation learning, we propose novel pretraining and finetuning methods which combine language understanding and recommendation tasks. Therefore, Recformer can effectively recommend the next item based on language representations. Extensive experiments conducted on six datasets demonstrate the effectiveness of Recformer for sequential recommendation, especially in low-resource and cold-start settings.