# Bidirectional Knowledge Distillation for Enhancing Sequential Recommendation with Large Language Models

arXiv: https://arxiv.org/abs/2505.18120 · HTML: https://mlsanity.com/paper/2505.18120/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for recommender systems, one of 12 directions in Knowledge distillation.
- Builds on 25 of its 48 references that are in the map.
- First posted on arXiv 23 May 2025.

## Placed in

- Knowledge distillation / [Distillation for recommender systems](https://mlsanity.com/distill/distillation-for-recommender-systems/)

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons](https://mlsanity.com/paper/1811.03233/) (2018)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems](https://mlsanity.com/paper/2310.09233/) (2023)
- [Filter-enhanced MLP is All You Need for Sequential Recommendation](https://mlsanity.com/paper/2202.13556/) (2022)
- [Leveraging Large Language Models for Sequential Recommendation](https://mlsanity.com/paper/2309.09261/) (2023)
- [LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation](https://mlsanity.com/paper/2405.20646/) (2024)
- [Harnessing Large Language Models for Text-Rich Sequential Recommendation](https://mlsanity.com/paper/2403.13325/) (2024)
- [Personalized Graph Signal Processing for Collaborative Filtering](https://mlsanity.com/paper/2302.02113/) (2023)
- [Large Language Models Enhanced Collaborative Filtering](https://mlsanity.com/paper/2403.17688/) (2024)
- [Aligning Large Language Models for Controllable Recommendations](https://mlsanity.com/paper/2403.05063/) (2024)
- [Mutual Distillation Learning Network for Trajectory-User Linking](https://mlsanity.com/paper/2205.03773/) (2022)
- [AutoSeqRec: Autoencoder for Efficient Sequential Recommendation](https://mlsanity.com/paper/2308.06878/) (2023)
- [Triple Structural Information Modelling for Accurate, Explainable and Interactive Recommendation](https://mlsanity.com/paper/2304.11528/) (2023)
- [Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks](https://mlsanity.com/paper/1706.07684/) (2017)
- [Adaptive In-Context Learning with Large Language Models for Bundle Generation](https://mlsanity.com/paper/2312.16262/) (2023)
- [Active Large Language Model-based Knowledge Distillation for Session-based Recommendation](https://mlsanity.com/paper/2502.15685/) (2024)

## Built on by (most cited first, 4 of 4)

- [From Hidden Profiles to Governable Personalization: Recommender Systems in the Age of LLM Agents](https://mlsanity.com/paper/2604.20065/) (2026)
- [Distribution-Aware End-to-End Embedding for Streaming Numerical Features in Click-Through Rate Prediction](https://mlsanity.com/paper/2602.03223/) (2026)
- [GOD: Enhancing Generalization via Deep Grafting for Sequential Recommendation](https://mlsanity.com/paper/2608.16073/) (2026)
- [SelfDR: Self-Distillation from Reasoning for LLM-Based Recommendation](https://mlsanity.com/paper/2609.03313/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated exceptional performance in understanding and generating semantic patterns, making them promising candidates for sequential recommendation tasks. However, when combined with conventional recommendation models (CRMs), LLMs often face challenges related to high inference costs and static knowledge transfer methods. In this paper, we propose a novel mutual distillation framework, LLMD4Rec, that fosters dynamic and bidirectional knowledge exchange between LLM-centric and CRM-based recommendation systems. Unlike traditional unidirectional distillation methods, LLMD4Rec enables iterative optimization by alternately refining both models, enhancing the semantic understanding of CRMs and enriching LLMs with collaborative signals from user-item interactions. By leveraging sample-wise adaptive weighting and aligning output distributions, our approach eliminates the need for additional parameters while ensuring effective knowledge transfer. Extensive experiments on real-world datasets demonstrate that LLMD4Rec significantly improves recommendation accuracy across multiple benchmarks without increasing inference costs. This method provides a scalable and efficient solution for combining the strengths of both LLMs and CRMs in sequential recommendation systems.