mlsanityData to 7 Oct 2026

Can Small Language Models be Good Reasoners for Sequential Recommendation?

arXiv 2403.04260 ↗First posted 7 Mar 2024cs.IR cs.CL cs.LGcited by 25

On the map: LLM reasoning → LLM reasoning for recommendation

Abstract (arXiv)

Large language models (LLMs) open up new horizons for sequential recommendations, owing to their remarkable language comprehension and generation capabilities. However, there are still numerous challenges that should be addressed to successfully implement sequential recommendations empowered by LLMs. Firstly, user behavior patterns are often complex, and relying solely on one-step reasoning from LLMs may lead to incorrect or task-irrelevant responses. Secondly, the prohibitively resource requirements of LLM (e.g., ChatGPT-175B) are overwhelmingly high and impractical for real sequential recommender systems. In this paper, we propose a novel Step-by-step knowLedge dIstillation fraMework for recommendation (SLIM), paving a promising path for sequential recommenders to enjoy the exceptional reasoning capabilities of LLMs in a "slim" (i.e., resource-efficient) manner. We introduce CoT prompting based on user behavior sequences for the larger teacher model. The rationales generated by the teacher model are then utilized as labels to distill the downstream smaller student model (e.g., LLaMA2-7B). In this way, the student model acquires the step-by-step reasoning capabilities in recommendation tasks. We encode the generated rationales from the student model into a dense vector, which empowers recommendation in both ID-based and ID-agnostic scenarios. Extensive experiments demonstrate the effectiveness of SLIM over state-of-the-art baselines, and further analysis showcasing its ability to generate meaningful recommendation reasoning at affordable costs.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in LLM reasoning for recommendation, one of 17 directions in LLM reasoning. Builds on 31 of its 41 references that are in the map. First posted on arXiv 7 Mar 2024.

Builds on

31 of 41 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2018Self-Attentive Sequential Recommendationcited by 1,169
  5. 2017Neural Collaborative Filteringcited by 1,108
  6. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 952
  7. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 710
  8. 2015Session-based Recommendations with Recurrent Neural Networkscited by 685
  9. 2018Personalized Top-N Sequential Recommendation via Convolutional Sequence Embeddingcited by 473
  10. 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 415
  11. 2023A Survey on Large Language Models for Recommendationcited by 378
  12. 2018Session-based Recommendation with Graph Neural Networkscited by 312
19 more
  1. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 307
  2. 2022Teaching Small Language Models to Reasoncited by 279
  3. 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 262
  4. 2023ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPTcited by 250
  5. 2023Is ChatGPT a Good Recommender? A Preliminary Studycited by 192
  6. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 188
  7. 2023Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approachcited by 168
  8. 2023Uncovering ChatGPT's Capabilities in Recommender Systemscited by 160
  9. 2023Text Is All You Need: Learning Language Representations for Sequential Recommendationcited by 149
  10. 2023How Can Recommender Systems Benefit from Large Language Models: A Surveycited by 141
  11. 2020RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithmscited by 137
  12. 2022Towards Universal Sequence Representation Learning for Recommender Systemscited by 120
  13. 2019Sequential Recommender Systems: Challenges, Progress and Prospectscited by 115
  14. 2023Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Predictioncited by 113
  15. 2023Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendationcited by 113
  16. 2023ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendationcited by 76
  17. 2018Sequence-Aware Recommender Systemscited by 58
  18. 2023A Preliminary Evaluation of ChatGPT for Zero-shot Dialogue Understandingcited by 31
  19. 2021Elliot: a Comprehensive and Rigorous Framework for Reproducible Recommender Systems Evaluationcited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 12; 2026: 5.

25 papers.

Sort:

By area:

  1. 2025Reason4Rec: Deliberative User Preference Alignment of Large Language Models for Recommendationcited by 21
  2. 2024Language Representations Can be What Recommenders Need: Findings and Potentialscited by 18
  3. 2025A Survey of Foundation Model-Powered Recommender Systems: From Feature-Based, Generative to Agentic Paradigmscited by 17
  4. 2025R2ec: Towards Large Recommender Models with Reasoningcited by 13
  5. 2024Efficient Inference for Large Language Model-based Generative Recommendationcited by 11
  6. 2024LLM4DSR: Leveraging Large Language Model for Denoising Sequential Recommendationcited by 10
  7. 2024Large Language Model Enhanced Recommender Systems: A Surveycited by 10
  8. 2025Interactive Recommendation Agent with Active User Commandscited by 10
  9. 2025Improving LLM-powered Recommendations with Personalized Informationcited by 9
  10. 2024Token-level Proximal Policy Optimization for Query Generationcited by 7
  11. 2024Active Large Language Model-based Knowledge Distillation for Session-based Recommendationcited by 6
  12. 2025Semantic Retrieval Augmented Contrastive Learning for Sequential Recommendationcited by 6
13 more
  1. 2024Reasoning over User Preferences: Knowledge Graph-Augmented LLMs for Explainable Conversational Recommendationscited by 4
  2. 2025Field Matters: A Lightweight LLM-enhanced Method for CTR Predictioncited by 4
  3. 2024Balancing Efficiency and Effectiveness: An LLM-Infused Approach for Optimized CTR Predictioncited by 3
  4. 2025Adaptive political surveys and GPT-4: Tackling the cold start problem with simulated user interactionscited by 3
  5. 2025Temporal User Profiling with LLMs: Balancing Short-Term and Long-Term Preferences for Recommendationscited by 2
  6. 2025Does Knowledge Distillation Matter for Large Language Model based Bundle Generation?cited by 1
  7. 2025Routing Distilled Knowledge via Mixture of LoRA Experts for Large Language Model based Bundle Generationcited by 1
  8. 2025SmallPlan: Leverage Small Language Models for Sequential Path Planning with Simulation-Powered, LLM-Guided Distillationcited by 0
  9. 2026VLM2Rec: Resolving Modality Collapse in Vision-Language Model Embedders for Multimodal Sequential Recommendationcited by 0
  10. 2026Can LLMs Fool Graph Learning? Exploring Universal Adversarial Attacks on Text-Attributed Graphscited by 0
  11. 2026HORIZON: A Benchmark for In-the-wild User Behaviour Modelingcited by 0
  12. 2026Modular Representation Compression: Adapting LLMs for Efficient and Effective Recommendationscited by 0
  13. 2026GOD: Enhancing Generalization via Deep Grafting for Sequential Recommendationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation
  2. 2024Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Model
  3. 2024GOT4Rec: Graph of Thoughts for Sequential Recommendation
  4. 2024Unleash LLMs Potential for Sequential Recommendation by Coordinating Dual Dynamic Index Mechanism
  5. 2024Exploring the Impact of Large Language Models on Recommender Systems: An Extensive Review
  6. 2024Rethinking Large Language Model Architectures for Sequential Recommendations
  7. 2026SelfDR: Self-Distillation from Reasoning for LLM-Based Recommendation
  8. 2023DRDT: Dynamic Reflection with Divergent Thinking for LLM-based Sequential Recommendation