BAHSD: Bridging the Long-tail Gap via Adaptive Distillation in Black-box Sequential Recommendation
Not on the map. Why?
Abstract (arXiv)
Sequential recommendation systems are widely adopted but often deployed as black-box APIs, which has driven recent interest in model extraction to replicate their capabilities locally. However, the long-tail distribution induces severe signal heterogeneity: dense head sequences trigger the solidification of teacher preference, biasing extraction toward local patterns, while sparse tail sequences yield flat, noisy predictions. Existing one-size-fits-all extraction overlooks this disparity, resulting in noise overfitting and suboptimal knowledge transfer. We propose BAHSD, a black-box adaptive distillation framework that handles signal heterogeneity via a multi-scale consistency probing mechanism to implicitly quantify signal reliability. Based on this, an adaptive hierarchical objective is designed: dynamic-temperature KL divergence mitigates preference solidification for high-confidence signals, while ranking consistency and InfoNCE contrastive learning provide noise-robust enhancement for low-confidence signals. BAHSD consistently outperforms baselines, achieving up to 4.98% gain over the teacher and 80%+ improvement on tail users, offering a plug-and-play solution for high-fidelity black-box recommendation extraction.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 39 references that are in the map. First posted on arXiv 2 Jun 2026.
Builds on
20 of 39 references.
References · 20
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2020Knowledge Distillation: A Surveycited by 1,578
- 2018Self-Attentive Sequential Recommendationcited by 1,169
- 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 952
- 2021Causal Intervention for Leveraging Popularity Bias in Recommendationcited by 116
- 2020Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender Systemcited by 94
- 2024CoRAL: Collaborative Retrieval-Augmented Large Language Models Improve Long-tail Recommendationcited by 44
- 2020A Model of Two Tales: Dual Transfer Learning Framework for Improved Long-tail Item Recommendationcited by 38
- 2021Black-Box Attacks on Sequential Recommenders via Data-Free Model Extractioncited by 34
- 2020DE-RRD: A Knowledge Distillation Framework for Recommender Systemcited by 28
- 2021Bidirectional Distillation for Top-K Recommender Systemcited by 21
- 2025ABKD: Pursuing a Proper Allocation of the Probability Mass in Knowledge Distillation via α-β-Divergencecited by 21
8 more
- 2024Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Modelcited by 13
- 2022Unbiased Knowledge Distillation for Recommendationcited by 11
- 2024Knowledge Distillation with Refined Logitscited by 9
- 2023Ensemble Modeling with Contrastive Knowledge Distillation for Sequential Recommendationcited by 6
- 2024Dual-Head Knowledge Distillation: Enhancing Logits Utilization with an Auxiliary Headcited by 4
- 2024Exploring Feature-based Knowledge Distillation for Recommender System: A Frequency Perspectivecited by 3
- 2025Rejuvenating Cross-Entropy Loss in Knowledge Distillation for Recommender Systemscited by 1
- 2026A Cognitive Distribution and Behavior-Consistent Framework for Black-Box Attacks on Recommender Systemscited by 1
Built on by
Not cited yet.