mlsanityData to 7 Oct 2026

Rethinking Large Language Model Architectures for Sequential Recommendations

arXiv 2402.09543 ↗First posted 14 Feb 2024cs.IRcited by 27

On the map: LLM reasoning → LLM reasoning for recommendation

Abstract (arXiv)

Recently, sequential recommendation has been adapted to the LLM paradigm to enjoy the power of LLMs. LLM-based methods usually formulate recommendation information into natural language and the model is trained to predict the next item in an auto-regressive manner. Despite their notable success, the substantial computational overhead of inference poses a significant obstacle to their real-world applicability. In this work, we endeavor to streamline existing LLM-based recommendation models and propose a simple yet highly effective model Lite-LLM4Rec. The primary goal of Lite-LLM4Rec is to achieve efficient inference for the sequential recommendation task. Lite-LLM4Rec circumvents the beam search decoding by using a straight item projection head for ranking scores generation. This design stems from our empirical observation that beam search decoding is ultimately unnecessary for sequential recommendations. Additionally, Lite-LLM4Rec introduces a hierarchical LLM structure tailored to efficiently handle the extensive contextual information associated with items, thereby reducing computational overhead while enjoying the capabilities of LLMs. Experiments on three publicly available datasets corroborate the effectiveness of Lite-LLM4Rec in both performance and inference efficiency (notably 46.8% performance improvement and 97.28% efficiency improvement on ML-1m) over existing LLM-based methods. Our implementations will be open sourced.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in LLM reasoning for recommendation, one of 17 directions in LLM reasoning. Builds on 23 of its 48 references that are in the map. First posted on arXiv 14 Feb 2024.

Builds on

23 of 48 references.

References · 23

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2020Longformer: The Long-Document Transformercited by 3,362
  5. 2015Session-based Recommendations with Recurrent Neural Networkscited by 685
  6. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 356
  7. 2023TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendationcited by 336
  8. 2023Is ChatGPT a Good Recommender? A Preliminary Studycited by 192
  9. 2023Recommender Systems with Generative Retrievalcited by 191
  10. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 188
  11. 2023Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approachcited by 168
  12. 2023Text Is All You Need: Learning Language Representations for Sequential Recommendationcited by 149
11 more
  1. 2023Large language models can accurately predict searcher preferencescited by 129
  2. 2023Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisitedcited by 124
  3. 2023Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Predictioncited by 113
  4. 2023How to Index Item IDs for Recommendation Foundation Modelscited by 102
  5. 2023LlamaRec: Two-Stage Recommendation using Large Language Models for Rankingcited by 52
  6. 2021Zero-Shot Recommender Systemscited by 45
  7. 2023E4SRec: An Elegant Effective Efficient Extensible Solution of Large Language Models for Sequential Recommendationcited by 39
  8. 2023Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insightscited by 38
  9. 2023Large Language Model Can Interpret Latent Space of Sequential Recommendercited by 19
  10. 2023LightLM: A Lightweight Deep and Narrow Language Model for Generative Recommendationcited by 8
  11. 2023ControlRec: Bridging the Semantic Gap between Language Model and Personalized Recommendationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 8; 2026: 11.

27 papers.

Sort:

By area:

  1. 2024LLM-Powered User Simulator for Recommender Systemcited by 38
  2. 2024TokenRec: Learning to Tokenize ID for LLM-based Generative Recommendationcited by 35
  3. 2024Graph Machine Learning in the Era of Large Language Models (LLMs)cited by 33
  4. 2024Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyondcited by 26
  5. 2024SLMRec: Distilling Large Language Models into Small for Sequential Recommendationcited by 18
  6. 2025Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expressioncited by 16
  7. 2025R4ec: A Reasoning, Reflection, and Refinement Framework for Recommendation Systemscited by 16
  8. 2024SIGMA: Selective Gated Mamba for Sequential Recommendationcited by 10
  9. 2024Semantic Convergence: Harmonizing Recommender Systems via Two-Stage Alignment and Behavioral Semantic Tokenizationcited by 9
  10. 2025Privacy Risks of LLM-Empowered Recommender Systems: An Inversion Attack Perspectivecited by 6
  11. 2024Pre-train, Align, and Disentangle: Empowering Sequential Recommendation with Large Language Modelscited by 4
  12. 2025GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networkscited by 3
15 more
  1. 2026Rethinking Recommendation Paradigms: From Pipelines to Agentic Recommender Systemscited by 3
  2. 2025LLMs for Supply Chain Managementcited by 2
  3. 2025Empowering Large Language Model for Sequential Recommendation via Multimodal Embeddings and Semantic IDscited by 2
  4. 2025LLM-EDT: Large Language Model Enhanced Cross-domain Sequential Recommendation with Dual-phase Trainingcited by 1
  5. 2026Masked Diffusion Generative Recommendationcited by 1
  6. 2026De-conflating Preference and Qualification: Constrained Dual-Perspective Reasoning for Job Recommendation with Large Language Modelscited by 1
  7. 2026EvoRec: Self Evolving Agentic Recommender Systemscited by 1
  8. 2025From What to Why: Thought-Space Recommendation with Small Language Modelscited by 0
  9. 2026Detecting Miscitation on the Scholarly Web through LLM-Augmented Text-Rich Graph Learningcited by 0
  10. 2026VLM4Rec: Multimodal Semantic Representation for Recommendation with Large Vision-Language Modelscited by 0
  11. 2026AgenticRS-Architecture: System Design for Agentic Recommender Systemscited by 0
  12. 2026DynamicPO: Dynamic Preference Optimization for Recommendationcited by 0
  13. 2026Stochastic Primal-Dual Decoding for Multiobjective Generative Recommender Systemscited by 0
  14. 2026Empowering Cross-Domain Sequential Recommendation with Hybrid Tokenization and Serial-Parallel Decodingcited by 0
  15. 2026ReMem: Rethinking Perception and Memory in Long-Context Recommendation Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation
  2. 2026TSPORec: Token Selection via Preference Optimization for LLM-Based Sequential Recommendation
  3. 2025Language Ranker: A Lightweight Ranking framework for LLM Decoding
  4. 2024Unleash LLMs Potential for Sequential Recommendation by Coordinating Dual Dynamic Index Mechanism
  5. 2025Rethinking LLM-Based Recommendations: A Personalized Query-Driven Parallel Integration
  6. 2024CALRec: Contrastive Alignment of Generative LLMs for Sequential Recommendation
  7. 2025GR-LLMs: Recent Advances in Generative Recommendation Based on Large Language Models
  8. 2024LLM4DSR: Leveraging Large Language Model for Denoising Sequential Recommendation