mlsanityData to 14 Oct 2026

Distillation for recommender systems

34 papers0 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: recommendation, knowledge, user, recommender, sequential

Key facts. Distillation for recommender systems: 34 papers on arXiv since 2021 (data through 14 Oct 2026). 9 of its 34 mapped papers (26%) cite “Topology Distillation for Recommender System” (2021, arXiv 2106.08700).

What’s new

Nothing new since 25 Sep. Latest: 4 Sep 2026.

Foundations

Most cited by papers here

  1. Topology Distillation for Recommender System2021cited by 9 here
  2. Dual Correction Strategy for Ranking Distillation in Top-N Recommender System2021cited by 6 here
  3. Distillation from Heterogeneous Models for Top-K Recommendation2023cited by 5 here
  4. Unbiased Knowledge Distillation for Recommendation2022cited by 4 here
  5. Can Small Language Models be Good Reasoners for Sequential Recommendation?2024cited by 2 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2021Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspectivecited by 1 here
  2. 2022Cross-Task Knowledge Distillation in Multi-Task Recommendationcited by 1 here
  3. 2022Unbiased Knowledge Distillation for Recommendationcited by 4 here
  4. 2024Exploring Feature-based Knowledge Distillation for Recommender System: A Frequency Perspectivecited by 1 here
20 more it builds on, not in this chain
  1. 2021Show, Attend and Distill:Knowledge Distillation via Attention-based Feature Matchingcited by 1 here
  2. 2021Adaptive Multi-Teacher Multi-level Knowledge Distillationcited by 2 here
  3. 2021Towards Understanding Knowledge Distillationcited by 1 here
  4. 2021One Teacher is Enough? Pre-trained Language Model Distillation from Multiple Teacherscited by 2 here
  5. 2021Topology Distillation for Recommender Systemcited by 9 here
  6. 2021Knowledge Distillation via Instance-level Sequence Learningcited by 1 here
  7. 2021Scene-adaptive Knowledge Distillation for Sequential Recommendation via Differentiable Architecture Searchcited by 1 here
  8. 2021Follow Your Path: a Progressive Method for Knowledge Distillationcited by 2 here
  9. 2021Dual Correction Strategy for Ranking Distillation in Top-N Recommender Systemcited by 6 here
  10. 2022Knowledge Distillation from A Stronger Teachercited by 2 here
  11. 2023Distillation from Heterogeneous Models for Top-K Recommendationcited by 5 here
  12. 2023SCOTT: Self-Consistent Chain-of-Thought Distillationcited by 1 here
  13. 2023Decoupled Kullback-Leibler Divergence Losscited by 1 here
  14. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 1 here
  15. 2024A Teacher-Free Graph Knowledge Distillation Framework with Dual Self-Distillationcited by 1 here
  16. 2024Can Small Language Models be Good Reasoners for Sequential Recommendation?cited by 2 here
  17. 2024Leave No One Behind: Online Self-Supervised Self-Distillation for Sequential Recommendationcited by 1 here
  18. 2024Knowledge Distillation Approaches for Accurate and Efficient Recommender Systemcited by 1 here
  19. 2024Bridging the Gap: Unpacking the Hidden Challenges in Knowledge Distillation for Online Ranking Systemscited by 1 here
  20. 2024Active Large Language Model-based Knowledge Distillation for Session-based Recommendationcited by 1 here

The 24 most cited of 28.

Browse

29 of 34, ranked 6–34

  1. 6Bidirectional Knowledge Distillation for Enhancing Sequential Recommendation with Large Language Models20251
  2. 7Leave No One Behind: Online Self-Supervised Self-Distillation for Sequential Recommendation20241
  3. 8Knowledge Distillation Approaches for Accurate and Efficient Recommender System20241
  4. 9Bridging the Gap: Unpacking the Hidden Challenges in Knowledge Distillation for Online Ranking Systems20241
  5. 10Exploring Feature-based Knowledge Distillation for Recommender System: A Frequency Perspective20241
  6. 11Active Large Language Model-based Knowledge Distillation for Session-based Recommendation20241
  7. 12Cross-Task Knowledge Distillation in Multi-Task Recommendation20221
  8. 13Scene-adaptive Knowledge Distillation for Sequential Recommendation via Differentiable Architecture Search20211
  9. 14Efficient Personalized Reranking with Semi-Autoregressive Generation and Online Knowledge Distillation20260
  10. 15Pre-trained LLMs Meet Sequential Recommenders: Efficient User-Centric Knowledge Distillation20260
All 29
  1. 16LoopFM: Learning frOm HistOrical RePresentations of Foundation Model for Recommendation20260
  2. 17GR2 Technical Report20260
  3. 18Diffusion-GR2: Diffusion Generative Reasoning Re-ranker20260
  4. 19GOD: Enhancing Generalization via Deep Grafting for Sequential Recommendation20260
  5. 20rEDMRec: Distilling Large Language Model Reasoning into an Editable Experience Memory for Recommendation20260
  6. 21Distill Globally, Adapt Locally: Reasoning Distillation and Product-Type Test-Time Training for Scalable Trade-Up Recommendation20260
  7. 22LaViC: Adapting Large Vision-Language Models to Visually-Aware Conversational Recommendation20250
  8. 23Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deployment20250
  9. 24Rejuvenating Cross-Entropy Loss in Knowledge Distillation for Recommender Systems20250
  10. 25BPL: Bias-adaptive Preference Distillation Learning for Recommender System20250
  11. 26CLLMRec: LLM-powered Cognitive-Aware Concept Recommendation via Semantic Alignment and Prerequisite Knowledge Distillation20250
  12. 27Enhancing Scalability in Recommender Systems through Lottery Ticket Hypothesis and Knowledge Distillation-based Neural Network Pruning20240
  13. 28SLMRec: Distilling Large Language Models into Small for Sequential Recommendation20240
  14. 29Pre-trained Language Model and Knowledge Distillation for Lightweight Sequential Recommendation20240
  15. 30Entire-Space Variational Information Exploitation for Post-Click Conversion Rate Prediction20240
  16. 31Spatial-Temporal Knowledge Distillation for Takeaway Recommendation20240
  17. 32Towards a Unified Conversational Recommendation System: Multi-task Learning via Contextualized Knowledge Distillation20230
  18. 33Rethinking Position Bias Modeling with Knowledge Distillation for CTR Prediction20220
  19. 34Directed Acyclic Graph Factorization Machines for CTR Prediction via Knowledge Distillation20220