mlsanityData to 7 Oct 2026

BAHSD: Bridging the Long-tail Gap via Adaptive Distillation in Black-box Sequential Recommendation

arXiv 2606.03091 ↗First posted 2 Jun 2026cs.IR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Sequential recommendation systems are widely adopted but often deployed as black-box APIs, which has driven recent interest in model extraction to replicate their capabilities locally. However, the long-tail distribution induces severe signal heterogeneity: dense head sequences trigger the solidification of teacher preference, biasing extraction toward local patterns, while sparse tail sequences yield flat, noisy predictions. Existing one-size-fits-all extraction overlooks this disparity, resulting in noise overfitting and suboptimal knowledge transfer. We propose BAHSD, a black-box adaptive distillation framework that handles signal heterogeneity via a multi-scale consistency probing mechanism to implicitly quantify signal reliability. Based on this, an adaptive hierarchical objective is designed: dynamic-temperature KL divergence mitigates preference solidification for high-confidence signals, while ranking consistency and InfoNCE contrastive learning provide noise-robust enhancement for low-confidence signals. BAHSD consistently outperforms baselines, achieving up to 4.98% gain over the teacher and 80%+ improvement on tail users, offering a plug-and-play solution for high-fidelity black-box recommendation extraction.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 39 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

20 of 39 references.

References · 20

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2020Knowledge Distillation: A Surveycited by 1,578
  3. 2018Self-Attentive Sequential Recommendationcited by 1,169
  4. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 952
  5. 2021Causal Intervention for Leveraging Popularity Bias in Recommendationcited by 116
  6. 2020Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender Systemcited by 94
  7. 2024CoRAL: Collaborative Retrieval-Augmented Large Language Models Improve Long-tail Recommendationcited by 44
  8. 2020A Model of Two Tales: Dual Transfer Learning Framework for Improved Long-tail Item Recommendationcited by 38
  9. 2021Black-Box Attacks on Sequential Recommenders via Data-Free Model Extractioncited by 34
  10. 2020DE-RRD: A Knowledge Distillation Framework for Recommender Systemcited by 28
  11. 2021Bidirectional Distillation for Top-K Recommender Systemcited by 21
  12. 2025ABKD: Pursuing a Proper Allocation of the Probability Mass in Knowledge Distillation via α-β-Divergencecited by 21
8 more
  1. 2024Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Modelcited by 13
  2. 2022Unbiased Knowledge Distillation for Recommendationcited by 11
  3. 2024Knowledge Distillation with Refined Logitscited by 9
  4. 2023Ensemble Modeling with Contrastive Knowledge Distillation for Sequential Recommendationcited by 6
  5. 2024Dual-Head Knowledge Distillation: Enhancing Logits Utilization with an Auxiliary Headcited by 4
  6. 2024Exploring Feature-based Knowledge Distillation for Recommender System: A Frequency Perspectivecited by 3
  7. 2025Rejuvenating Cross-Entropy Loss in Knowledge Distillation for Recommender Systemscited by 1
  8. 2026A Cognitive Distribution and Behavior-Consistent Framework for Black-Box Attacks on Recommender Systemscited by 1

Built on by

Not cited yet.