# Distillation-based Scenario-Adaptive Mixture-of-Experts for the Matching Stage of Multi-scenario Recommendation

arXiv: https://arxiv.org/abs/2601.02368 · HTML: https://mlsanity.com/paper/2601.02368/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MoE for recommender systems, one of 12 directions in Mixture-of-experts.
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 28 Nov 2025.

## Placed in

- Mixture-of-experts / [MoE for recommender systems](https://mlsanity.com/moe/moe-for-recommender-systems/)

## Builds on (most cited first, 18 of 18)

- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Embedding-based Retrieval in Facebook Search](https://mlsanity.com/paper/2006.11632/) (2020)
- [Multi-Interest Network with Dynamic Routing for Recommendation at Tmall](https://mlsanity.com/paper/1904.08030/) (2019)
- [KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos](https://mlsanity.com/paper/2208.08696/) (2022)
- [PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information](https://mlsanity.com/paper/2302.01115/) (2023)
- [M3oE: Multi-Domain Multi-Task Mixture-of Experts Recommendation Framework](https://mlsanity.com/paper/2404.18465/) (2024)
- [SAR-Net: A Scenario-Aware Ranking Network for Personalized Fair Recommendation in Hundreds of Travel Scenarios](https://mlsanity.com/paper/2110.06475/) (2021)
- [HAMUR: Hyper Adapter for Multi-Domain Recommendation](https://mlsanity.com/paper/2309.06217/) (2023)
- [Adaptive Domain Interest Network for Multi-domain Recommendation](https://mlsanity.com/paper/2206.09672/) (2022)
- [Learning Piece-wise Linear Models from Large Scale Data for Ad Click Prediction](https://mlsanity.com/paper/1704.05194/) (2017)
- [HiNet: Novel Multi-Scenario & Multi-Task Learning with Hierarchical Information Extraction](https://mlsanity.com/paper/2303.06095/) (2023)
- [Scenario-Adaptive and Self-Supervised Model for Multi-Scenario Personalized Recommendation](https://mlsanity.com/paper/2208.11457/) (2022)
- [Multi-Scenario Ranking with Adaptive Feature Learning](https://mlsanity.com/paper/2306.16732/) (2023)
- [ColdNAS: Search to Modulate for User Cold-Start Recommendation](https://mlsanity.com/paper/2306.03387/) (2023)
- [Knowledge-Enhanced Recommendation with User-Centric Subgraph Network](https://mlsanity.com/paper/2403.14377/) (2024)
- [M-scan: A Multi-Scenario Causal-driven Adaptive Network for Recommendation](https://mlsanity.com/paper/2404.07581/) (2024)
- [PERSCEN: Learning Personalized Interaction Pattern and Scenario Preference for Multi-Scenario Matching](https://mlsanity.com/paper/2506.18382/) (2025)

## Abstract (arXiv)

Multi-scenario recommendation is pivotal for optimizing user experience across diverse contexts. While Multi-gate Mixture-of-Experts (MMOE) thrives in ranking, its transfer to the matching stage is hindered by the blind optimization inherent to independent two-tower architectures and the parameter dominance of head scenarios. To address these structural and distributional bottlenecks, we propose Distillation-based Scenario-Adaptive Mixture-of-Experts (DSMOE). Specially, we devise a Scenario-Adaptive Projection (SAP) module to generate lightweight, context-specific parameters, effectively preventing expert collapse in long-tail scenarios. Concurrently, we introduce a cross-architecture knowledge distillation framework, where an interaction-aware teacher guides the two-tower student to capture complex matching patterns. Extensive experiments on real-world datasets demonstrate DSMOE's superiority, particularly in significantly improving retrieval quality for under-represented, data-sparse scenarios.