# Automatic Expert Selection for Multi-Scenario and Multi-Task Search

arXiv: https://arxiv.org/abs/2205.14321 · HTML: https://mlsanity.com/paper/2205.14321/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 28 May 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate](https://mlsanity.com/paper/1804.07931/) (2018)
- [One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Prediction](https://mlsanity.com/paper/2101.11427/) (2021)
- [Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising](https://mlsanity.com/paper/2105.08489/) (2021)
- [A Unified Perspective on Multi-Domain and Multi-Task Learning](https://mlsanity.com/paper/1412.7489/) (2014)
- [Latent Multi-task Architecture Learning](https://mlsanity.com/paper/1705.08142/) (2017)
- [M6-T: Exploring Sparse Expert Models and Beyond](https://mlsanity.com/paper/2105.15082/) (2021)
- [Scenario-aware and Mutual-based approach for Multi-scenario Recommendation in E-Commerce](https://mlsanity.com/paper/2012.08952/) (2020)
- [Unbiased Gradient Estimation with Balanced Assignments for Mixtures of Experts](https://mlsanity.com/paper/2109.11817/) (2021)
- [Adversarial Mixture Of Experts with Category Hierarchy Soft Constraint](https://mlsanity.com/paper/2007.12349/) (2020)

## Built on by (most cited first, 9 of 9)

- [Multi-granular Adversarial Attacks against Black-box Neural Ranking Models](https://mlsanity.com/paper/2404.01574/) (2024)
- [Multi-Scenario Ranking with Adaptive Feature Learning](https://mlsanity.com/paper/2306.16732/) (2023)
- [CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrieval](https://mlsanity.com/paper/2311.02834/) (2023)
- [BASM: A Bottom-up Adaptive Spatiotemporal Model for Online Food Ordering Service](https://mlsanity.com/paper/2211.12033/) (2022)
- [Deep Task-specific Bottom Representation Network for Multi-Task Recommendation](https://mlsanity.com/paper/2308.05996/) (2023)
- [Hybrid Contrastive Constraints for Multi-Scenario Ad Ranking](https://mlsanity.com/paper/2302.02636/) (2023)
- [ECAT: A Entire space Continual and Adaptive Transfer Learning Framework for Cross-Domain Recommendation](https://mlsanity.com/paper/2407.02542/) (2024)
- [Large-Scale Multi-Domain Recommendation: an Automatic Domain Feature Extraction and Personalized Integration Framework](https://mlsanity.com/paper/2404.08361/) (2024)
- [Scenario-Adaptive Fine-Grained Personalization Network: Tailoring User Behavior Representation to the Scenario Context](https://mlsanity.com/paper/2404.09709/) (2024)

## Abstract (arXiv)

Multi-scenario learning (MSL) enables a service provider to cater for users' fine-grained demands by separating services for different user sectors, e.g., by user's geographical region. Under each scenario there is a need to optimize multiple task-specific targets e.g., click through rate and conversion rate, known as multi-task learning (MTL). Recent solutions for MSL and MTL are mostly based on the multi-gate mixture-of-experts (MMoE) architecture. MMoE structure is typically static and its design requires domain-specific knowledge, making it less effective in handling both MSL and MTL. In this paper, we propose a novel Automatic Expert Selection framework for Multi-scenario and Multi-task search, named AESM^{2}. AESM^{2} integrates both MSL and MTL into a unified framework with an automatic structure learning. Specifically, AESM^{2} stacks multi-task layers over multi-scenario layers. This hierarchical design enables us to flexibly establish intrinsic connections between different scenarios, and at the same time also supports high-level feature extraction for different tasks. At each multi-scenario/multi-task layer, a novel expert selection algorithm is proposed to automatically identify scenario-/task-specific and shared experts for each input. Experiments over two real-world large-scale datasets demonstrate the effectiveness of AESM^{2} over a battery of strong baselines. Online A/B test also shows substantial performance gain on multiple metrics. Currently, AESM^{2} has been deployed online for serving major traffic.