mlsanityData to 14 Oct 2026

GOD: Enhancing Generalization via Deep Grafting for Sequential Recommendation

arXiv 2608.16073 ↗First posted 17 Aug 2026cs.IR cs.LGcited by 0

On the map: Knowledge distillation → Distillation for recommender systems

Abstract (arXiv)

Sequential recommenders often struggle with sparse and noisy histories, limiting generalization to unseen interactions. Knowledge distillation mitigates this by transferring dense supervision from a teacher to a student. However, most distillation methods run teacher and student independently, then match student outputs or representations to the teacher. Such supervision entangles student-component effects, blurring whether weak generalization stems from unreliable embeddings, overfitted encoding, or co-adaptation to sparse histories. In this paper, we propose Graft-Oriented Distillation (GOD), a component-level distillation framework for improved generalization through grafting. Grafting denotes replacing selected frozen-teacher components with trainable student counterparts to build hybrid source models. GOD uses these hybrid models to evaluate student embeddings with the teacher encoder and the student encoder with teacher embeddings, providing component-level feedback. At inference, GOD uses only the student, incurring no additional cost. Across three real-world datasets, GOD outperforms state-of-the-art baselines by up to 13.92%.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Distillation for recommender systems, one of 12 directions in Knowledge distillation. Builds on 48 of its 63 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

48 of 63 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Bidirectional Knowledge Distillation for Enhancing Sequential Recommendation with Large Language Models23 May
  2. 2024Leave No One Behind: Online Self-Supervised Self-Distillation for Sequential Recommendation22 Mar
  3. 2024Can Small Language Models be Good Reasoners for Sequential Recommendation?7 Mar
  4. 2021Dual Correction Strategy for Ranking Distillation in Top-N Recommender System8 Sep
  5. 2021Scene-adaptive Knowledge Distillation for Sequential Recommendation via Differentiable Architecture Search15 Jul
  6. 2021Topology Distillation for Recommender System16 Jun

Other references · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  7. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  8. 2018Self-Attentive Sequential Recommendationcited by 1,184
  9. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  10. 2015Image-based Recommendations on Styles and Substitutescited by 706
  11. 2017Deep Mutual Learningcited by 628
  12. 2018Personalized Top-N Sequential Recommendation via Convolutional Sequence Embeddingcited by 476
30 more
  1. 2020Circle Loss: A Unified Perspective of Pair Similarity Optimizationcited by 403
  2. 2019Multi-Similarity Loss with General Pair Weighting for Deep Metric Learningcited by 368
  3. 2020S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximizationcited by 311
  4. 2017Neural Attentive Session-based Recommendationcited by 237
  5. 2021Contrastive Learning for Representation Degeneration Problem in Sequential Recommendationcited by 121
  6. 2022Filter-enhanced MLP is All You Need for Sequential Recommendationcited by 96
  7. 2022Intent Contrastive Learning for Sequential Recommendationcited by 91
  8. 2016Improved Recurrent Neural Networks for Session-based Recommendationscited by 86
  9. 2016Yelp Dataset Challenge: Review Rating Predictioncited by 84
  10. 2020Denoising Implicit Feedback for Recommendationcited by 80
  11. 2018Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender Systemcited by 60
  12. 2021Contrastive Self-supervised Sequential Recommendation with Robust Augmentationcited by 56
  13. 2021Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformercited by 45
  14. 2020Progressive Network Grafting for Few-Shot Knowledge Distillationcited by 25
  15. 2024RDRec: Rationale Distillation for LLM-based Recommendationcited by 22
  16. 2021Bidirectional Distillation for Top-K Recommender Systemcited by 21
  17. 2024LinRec: Linear Attention Mechanism for Long-term Sequential Recommender Systemscited by 21
  18. 2020Solvable Model for Inheriting the Regularization through Knowledge Distillationcited by 16
  19. 2023Attention Calibration for Transformer-based Sequential Recommendationcited by 15
  20. 2019Collaborative Distillation for Top-N Recommendationcited by 13
  21. 2023Meta-optimized Contrastive Learning for Sequential Recommendationcited by 10
  22. 2023Intent Contrastive Learning with Cross Subsequences for Sequential Recommendationcited by 9
  23. 2020Filter Grafting for Deep Neural Networkscited by 8
  24. 2021Improving Sequential Recommendation Consistency with Self-Supervised Imitationcited by 7
  25. 2023Ensemble Modeling with Contrastive Knowledge Distillation for Sequential Recommendationcited by 6
  26. 2021CT4Rec: Simple yet Effective Consistency Training for Sequential Recommendationcited by 4
  27. 2025Capturing User Interests from Data Streams for Continual Sequential Recommendationcited by 3
  28. 2024Curriculum-scheduled Knowledge Distillation from Multiple Pre-trained Teachers for Multi-domain Sequential Recommendationcited by 1
  29. 2025Relative Contrastive Learning for Sequential Recommendation with Similarity-based Positive Pair Selectioncited by 1
  30. 2026From Overlooked to Explored: Recovering Item Relations via Mixture of Perspectives for Sequential Recommendationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Sparse Autoencoders for Sequential Recommendation Models: Interpretation and Flexible Control
  2. 2025Improving LLM Interpretability and Performance via Guided Embedding Refinement for Sequential Recommendation
  3. 2023Generate What You Prefer: Reshaping Sequential Recommendation via Guided Diffusion
  4. 2023Diffusion Augmentation for Sequential Recommendation
  5. 2024Pre-train, Align, and Disentangle: Empowering Sequential Recommendation with Large Language Models
  6. 2024Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Model
  7. 2025Realizing Scaling Laws in Recommender Systems: A Foundation-Expert Paradigm for Hyperscale Model Deployment
  8. 2026Fusion and Alignment Enhancement with Large Language Models for Tail-item Sequential Recommendation