# LLMEmb: Large Language Model Can Be a Good Embedding Generator for Sequential Recommendation

arXiv: https://arxiv.org/abs/2409.19925 · HTML: https://mlsanity.com/paper/2409.19925/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2024.

## Built on by (most cited first, 18 of 18)

- [Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expression](https://mlsanity.com/paper/2505.13527/) (2025)
- [Bridge the Domains: Large Language Models Enhanced Cross-domain Sequential Recommendation](https://mlsanity.com/paper/2504.18383/) (2025)
- [CoVE: Compressed Vocabulary Expansion Makes Better LLM-based Recommender Systems](https://mlsanity.com/paper/2506.19993/) (2025)
- [Model Merging for Knowledge Editing](https://mlsanity.com/paper/2506.12384/) (2025)
- [Empowering Large Language Model for Sequential Recommendation via Multimodal Embeddings and Semantic IDs](https://mlsanity.com/paper/2509.02017/) (2025)
- [Enhancing Time Awareness in Generative Recommendation](https://mlsanity.com/paper/2509.13957/) (2025)
- [SemaCDR: LLM-Powered Transferable Semantics for Cross-Domain Sequential Recommendation](https://mlsanity.com/paper/2604.09551/) (2026)
- [SemSR: Semantics aware robust Session-based Recommendations](https://mlsanity.com/paper/2508.20587/) (2025)
- [Fine-grained auxiliary learning for real-world product recommendation](https://mlsanity.com/paper/2510.04551/) (2025)
- [The Best of the Two Worlds: Harmonizing Semantic and Hash IDs for Sequential Recommendation](https://mlsanity.com/paper/2512.10388/) (2025)
- [U-MASK: User-adaptive Spatio-Temporal Masking for Personalized Mobile AI Applications](https://mlsanity.com/paper/2601.06867/) (2026)
- [Compass-Embedding v4: Robust Contrastive Learning for Multilingual E-commerce Embeddings](https://mlsanity.com/paper/2601.11565/) (2025)
- [VLM2Rec: Resolving Modality Collapse in Vision-Language Model Embedders for Multimodal Sequential Recommendation](https://mlsanity.com/paper/2603.17450/) (2026)
- [Causal Direct Preference Optimization for Distributionally Robust Generative Recommendation](https://mlsanity.com/paper/2603.22335/) (2026)
- [TME-PSR: Time-aware, Multi-interest, and Explanation Personalization for Sequential Recommendation](https://mlsanity.com/paper/2604.09439/) (2026)
- [Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation](https://mlsanity.com/paper/2604.21536/) (2026)
- [Bridging Textual Profiles and Latent User Embeddings for Personalization](https://mlsanity.com/paper/2605.06981/) (2026)
- [MGDiff: Multi-Interest Sequence Recommendation with Masking GNN-Guided Diffusion](https://mlsanity.com/paper/2609.01619/) (2026)

## Abstract (arXiv)

Sequential Recommender Systems (SRS), which model a user's interaction history to predict the next item of interest, are widely used in various applications. However, existing SRS often struggle with low-popularity items, a challenge known as the long-tail problem. This issue leads to reduced serendipity for users and diminished profits for sellers, ultimately harming the overall system. Large Language Model (LLM) has the ability to capture semantic relationships between items, independent of their popularity, making it a promising solution to this problem. In this paper, we introduce LLMEmb, a novel method leveraging LLM to generate item embeddings that enhance SRS performance. To bridge the gap between general-purpose LLM and the recommendation domain, we propose a Supervised Contrastive Fine-Tuning (SCFT) approach. This approach includes attribute-level data augmentation and a tailored contrastive loss to make LLM more recommendation-friendly. Additionally, we emphasize the importance of integrating collaborative signals into LLM-generated embeddings, for which we propose Recommendation Adaptation Training (RAT). This further refines the embeddings for optimal use in SRS. The LLMEmb-derived embeddings can be seamlessly integrated with any SRS models, underscoring the practical value. Comprehensive experiments conducted on three real-world datasets demonstrate that LLMEmb significantly outperforms existing methods across multiple SRS models. The code for our method is released online https://github.com/Applied-Machine-Learning-Lab/LLMEmb.