# BAHSD: Bridging the Long-tail Gap via Adaptive Distillation in Black-box Sequential Recommendation

arXiv: https://arxiv.org/abs/2606.03091 · HTML: https://mlsanity.com/paper/2606.03091/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 2 Jun 2026.

## Builds on (most cited first, 20 of 20)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Causal Intervention for Leveraging Popularity Bias in Recommendation](https://mlsanity.com/paper/2105.06067/) (2021)
- [Model-Agnostic Counterfactual Reasoning for Eliminating Popularity Bias in Recommender System](https://mlsanity.com/paper/2010.15363/) (2020)
- [CoRAL: Collaborative Retrieval-Augmented Large Language Models Improve Long-tail Recommendation](https://mlsanity.com/paper/2403.06447/) (2024)
- [A Model of Two Tales: Dual Transfer Learning Framework for Improved Long-tail Item Recommendation](https://mlsanity.com/paper/2010.15982/) (2020)
- [Black-Box Attacks on Sequential Recommenders via Data-Free Model Extraction](https://mlsanity.com/paper/2109.01165/) (2021)
- [DE-RRD: A Knowledge Distillation Framework for Recommender System](https://mlsanity.com/paper/2012.04357/) (2020)
- [ABKD: Pursuing a Proper Allocation of the Probability Mass in Knowledge Distillation via α-β-Divergence](https://mlsanity.com/paper/2505.04560/) (2025)
- [Bidirectional Distillation for Top-K Recommender System](https://mlsanity.com/paper/2106.02870/) (2021)
- [Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Model](https://mlsanity.com/paper/2405.00338/) (2024)
- [Unbiased Knowledge Distillation for Recommendation](https://mlsanity.com/paper/2211.14729/) (2022)
- [Knowledge Distillation with Refined Logits](https://mlsanity.com/paper/2408.07703/) (2024)
- [Ensemble Modeling with Contrastive Knowledge Distillation for Sequential Recommendation](https://mlsanity.com/paper/2304.14668/) (2023)
- [Dual-Head Knowledge Distillation: Enhancing Logits Utilization with an Auxiliary Head](https://mlsanity.com/paper/2411.08937/) (2024)
- [Exploring Feature-based Knowledge Distillation for Recommender System: A Frequency Perspective](https://mlsanity.com/paper/2411.10676/) (2024)
- [Rejuvenating Cross-Entropy Loss in Knowledge Distillation for Recommender Systems](https://mlsanity.com/paper/2509.20989/) (2025)
- [A Cognitive Distribution and Behavior-Consistent Framework for Black-Box Attacks on Recommender Systems](https://mlsanity.com/paper/2602.10633/) (2026)

## Abstract (arXiv)

Sequential recommendation systems are widely adopted but often deployed as black-box APIs, which has driven recent interest in model extraction to replicate their capabilities locally. However, the long-tail distribution induces severe signal heterogeneity: dense head sequences trigger the solidification of teacher preference, biasing extraction toward local patterns, while sparse tail sequences yield flat, noisy predictions. Existing one-size-fits-all extraction overlooks this disparity, resulting in noise overfitting and suboptimal knowledge transfer. We propose BAHSD, a black-box adaptive distillation framework that handles signal heterogeneity via a multi-scale consistency probing mechanism to implicitly quantify signal reliability. Based on this, an adaptive hierarchical objective is designed: dynamic-temperature KL divergence mitigates preference solidification for high-confidence signals, while ranking consistency and InfoNCE contrastive learning provide noise-robust enhancement for low-confidence signals. BAHSD consistently outperforms baselines, achieving up to 4.98% gain over the teacher and 80%+ improvement on tail users, offering a plug-and-play solution for high-fidelity black-box recommendation extraction.