# Rethinking Large Language Model Architectures for Sequential Recommendations

arXiv: https://arxiv.org/abs/2402.09543 · HTML: https://mlsanity.com/paper/2402.09543/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in LLM reasoning for recommendation, one of 17 directions in LLM reasoning.
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 14 Feb 2024.

## Placed in

- LLM reasoning / [LLM reasoning for recommendation](https://mlsanity.com/lr/llm-reasoning-for-recommendation/)

## Builds on (most cited first, 23 of 23)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents](https://mlsanity.com/paper/2304.09542/) (2023)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [Recommender Systems with Generative Retrieval](https://mlsanity.com/paper/2305.05065/) (2023)
- [Is ChatGPT a Good Recommender? A Preliminary Study](https://mlsanity.com/paper/2304.10149/) (2023)
- [Recommender Systems in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2307.02046/) (2023)
- [Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach](https://mlsanity.com/paper/2305.07001/) (2023)
- [Text Is All You Need: Learning Language Representations for Sequential Recommendation](https://mlsanity.com/paper/2305.13731/) (2023)
- [Large language models can accurately predict searcher preferences](https://mlsanity.com/paper/2309.10621/) (2023)
- [Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited](https://mlsanity.com/paper/2303.13835/) (2023)
- [Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction](https://mlsanity.com/paper/2305.06474/) (2023)
- [How to Index Item IDs for Recommendation Foundation Models](https://mlsanity.com/paper/2305.06569/) (2023)
- [LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking](https://mlsanity.com/paper/2311.02089/) (2023)
- [Zero-Shot Recommender Systems](https://mlsanity.com/paper/2105.08318/) (2021)
- [E4SRec: An Elegant Effective Efficient Extensible Solution of Large Language Models for Sequential Recommendation](https://mlsanity.com/paper/2312.02443/) (2023)
- [Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights](https://mlsanity.com/paper/2305.11700/) (2023)
- [Large Language Model Can Interpret Latent Space of Sequential Recommender](https://mlsanity.com/paper/2310.20487/) (2023)
- [LightLM: A Lightweight Deep and Narrow Language Model for Generative Recommendation](https://mlsanity.com/paper/2310.17488/) (2023)
- [ControlRec: Bridging the Semantic Gap between Language Model and Personalized Recommendation](https://mlsanity.com/paper/2311.16441/) (2023)

## Built on by (most cited first, 27 of 27)

- [LLM-Powered User Simulator for Recommender System](https://mlsanity.com/paper/2412.16984/) (2024)
- [TokenRec: Learning to Tokenize ID for LLM-based Generative Recommendation](https://mlsanity.com/paper/2406.10450/) (2024)
- [Graph Machine Learning in the Era of Large Language Models (LLMs)](https://mlsanity.com/paper/2404.14928/) (2024)
- [Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond](https://mlsanity.com/paper/2410.19744/) (2024)
- [SLMRec: Distilling Large Language Models into Small for Sequential Recommendation](https://mlsanity.com/paper/2405.17890/) (2024)
- [Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expression](https://mlsanity.com/paper/2505.13527/) (2025)
- [R4ec: A Reasoning, Reflection, and Refinement Framework for Recommendation Systems](https://mlsanity.com/paper/2507.17249/) (2025)
- [SIGMA: Selective Gated Mamba for Sequential Recommendation](https://mlsanity.com/paper/2408.11451/) (2024)
- [Semantic Convergence: Harmonizing Recommender Systems via Two-Stage Alignment and Behavioral Semantic Tokenization](https://mlsanity.com/paper/2412.13771/) (2024)
- [Privacy Risks of LLM-Empowered Recommender Systems: An Inversion Attack Perspective](https://mlsanity.com/paper/2508.03703/) (2025)
- [Pre-train, Align, and Disentangle: Empowering Sequential Recommendation with Large Language Models](https://mlsanity.com/paper/2412.04107/) (2024)
- [LLMs for Supply Chain Management](https://mlsanity.com/paper/2505.18597/) (2025)
- [GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networks](https://mlsanity.com/paper/2506.16114/) (2025)
- [Rethinking Recommendation Paradigms: From Pipelines to Agentic Recommender Systems](https://mlsanity.com/paper/2603.26100/) (2026)
- [Empowering Large Language Model for Sequential Recommendation via Multimodal Embeddings and Semantic IDs](https://mlsanity.com/paper/2509.02017/) (2025)
- [Masked Diffusion Generative Recommendation](https://mlsanity.com/paper/2601.19501/) (2026)
- [De-conflating Preference and Qualification: Constrained Dual-Perspective Reasoning for Job Recommendation with Large Language Models](https://mlsanity.com/paper/2602.03097/) (2026)
- [LLM-EDT: Large Language Model Enhanced Cross-domain Sequential Recommendation with Dual-phase Training](https://mlsanity.com/paper/2511.19931/) (2025)
- [EvoRec: Self Evolving Agentic Recommender Systems](https://mlsanity.com/paper/2606.28368/) (2026)
- [From What to Why: Thought-Space Recommendation with Small Language Models](https://mlsanity.com/paper/2510.08626/) (2025)
- [Detecting Miscitation on the Scholarly Web through LLM-Augmented Text-Rich Graph Learning](https://mlsanity.com/paper/2603.12290/) (2026)
- [VLM4Rec: Multimodal Semantic Representation for Recommendation with Large Vision-Language Models](https://mlsanity.com/paper/2603.12625/) (2026)
- [AgenticRS-Architecture: System Design for Agentic Recommender Systems](https://mlsanity.com/paper/2603.26085/) (2026)
- [DynamicPO: Dynamic Preference Optimization for Recommendation](https://mlsanity.com/paper/2605.00327/) (2026)
- [Stochastic Primal-Dual Decoding for Multiobjective Generative Recommender Systems](https://mlsanity.com/paper/2607.19357/) (2026)
- [Empowering Cross-Domain Sequential Recommendation with Hybrid Tokenization and Serial-Parallel Decoding](https://mlsanity.com/paper/2607.28659/) (2026)
- [ReMem: Rethinking Perception and Memory in Long-Context Recommendation Agents](https://mlsanity.com/paper/2609.37311/) (2026)

## Abstract (arXiv)

Recently, sequential recommendation has been adapted to the LLM paradigm to enjoy the power of LLMs. LLM-based methods usually formulate recommendation information into natural language and the model is trained to predict the next item in an auto-regressive manner. Despite their notable success, the substantial computational overhead of inference poses a significant obstacle to their real-world applicability. In this work, we endeavor to streamline existing LLM-based recommendation models and propose a simple yet highly effective model Lite-LLM4Rec. The primary goal of Lite-LLM4Rec is to achieve efficient inference for the sequential recommendation task. Lite-LLM4Rec circumvents the beam search decoding by using a straight item projection head for ranking scores generation. This design stems from our empirical observation that beam search decoding is ultimately unnecessary for sequential recommendations. Additionally, Lite-LLM4Rec introduces a hierarchical LLM structure tailored to efficiently handle the extensive contextual information associated with items, thereby reducing computational overhead while enjoying the capabilities of LLMs. Experiments on three publicly available datasets corroborate the effectiveness of Lite-LLM4Rec in both performance and inference efficiency (notably 46.8% performance improvement and 97.28% efficiency improvement on ML-1m) over existing LLM-based methods. Our implementations will be open sourced.