# M3oE: Multi-Domain Multi-Task Mixture-of Experts Recommendation Framework

arXiv: https://arxiv.org/abs/2404.18465 · HTML: https://mlsanity.com/paper/2404.18465/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in MoE for recommender systems, one of 12 directions in Mixture-of-experts.
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 29 Apr 2024.

## Placed in

- Mixture-of-experts / [MoE for recommender systems](https://mlsanity.com/moe/moe-for-recommender-systems/)

## Builds on (most cited first, 16 of 16)

- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Prediction](https://mlsanity.com/paper/2101.11427/) (2021)
- [DDTCDR: Deep Dual Transfer Cross Domain Recommendation](https://mlsanity.com/paper/1910.05189/) (2019)
- [PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information](https://mlsanity.com/paper/2302.01115/) (2023)
- [Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning](https://mlsanity.com/paper/2106.02705/) (2021)
- [MetaBalance: Improving Multi-Task Recommendations via Adapting Gradient Magnitudes of Auxiliary Tasks](https://mlsanity.com/paper/2203.06801/) (2022)
- [AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning](https://mlsanity.com/paper/2211.15055/) (2022)
- [A Survey on Cross-domain Recommendation: Taxonomies, Methods, and Future Directions](https://mlsanity.com/paper/2108.03357/) (2021)
- [Multi-Task Deep Recommender Systems: A Survey](https://mlsanity.com/paper/2302.03525/) (2023)
- [Multi-Task Recommendations with Reinforcement Learning](https://mlsanity.com/paper/2302.03328/) (2023)
- [SAR-Net: A Scenario-Aware Ranking Network for Personalized Fair Recommendation in Hundreds of Travel Scenarios](https://mlsanity.com/paper/2110.06475/) (2021)
- [HAMUR: Hyper Adapter for Multi-Domain Recommendation](https://mlsanity.com/paper/2309.06217/) (2023)
- [Leaving No One Behind: A Multi-Scenario Multi-Task Meta Learning Approach for Advertiser Modeling](https://mlsanity.com/paper/2201.06814/) (2022)
- [ERASE: Benchmarking Feature Selection Methods for Deep Recommender Systems](https://mlsanity.com/paper/2403.12660/) (2024)
- [AdaSparse: Learning Adaptively Sparse Structures for Multi-Domain Click-Through Rate Prediction](https://mlsanity.com/paper/2206.13108/) (2022)
- [Scenario-Aware Hierarchical Dynamic Network for Multi-Scenario Recommendation](https://mlsanity.com/paper/2309.02061/) (2023)

## Built on by (most cited first, 30 of 32)

- [X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model](https://mlsanity.com/paper/2510.10274/) (2025)
- [FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts](https://mlsanity.com/paper/2408.11304/) (2024)
- [MoE-Loco: Mixture of Experts for Multitask Locomotion](https://mlsanity.com/paper/2503.08564/) (2025)
- [Hierarchical Time-Aware Mixture of Experts for Multi-Modal Sequential Recommendation](https://mlsanity.com/paper/2501.14269/) (2025)
- [Bridge the Domains: Large Language Models Enhanced Cross-domain Sequential Recommendation](https://mlsanity.com/paper/2504.18383/) (2025)
- [Mixture of Experts in Large Language Models](https://mlsanity.com/paper/2507.11181/) (2025)
- [Personalize Before Retrieve: LLM-based Personalized Query Expansion for User-Centric Retrieval](https://mlsanity.com/paper/2510.08935/) (2025)
- [Hierarchical LoRA MoE for Efficient CTR Model Scaling](https://mlsanity.com/paper/2510.10432/) (2025)
- [Training-free LLM Merging for Multi-task Learning](https://mlsanity.com/paper/2506.12379/) (2025)
- [Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning](https://mlsanity.com/paper/2509.18942/) (2025)
- [Zenith: Scaling up Ranking Models for Billion-scale Livestreaming Recommendation](https://mlsanity.com/paper/2601.21285/) (2026)
- [Learning to Compose for Cross-domain Agentic Workflow Generation](https://mlsanity.com/paper/2602.11114/) (2026)
- [Empowering Large Language Model for Sequential Recommendation via Multimodal Embeddings and Semantic IDs](https://mlsanity.com/paper/2509.02017/) (2025)
- [Multimodal LLM With Hierarchical Mixture-of-Experts for VQA on 3D Brain MRI](https://mlsanity.com/paper/2509.25889/) (2025)
- [Personalized Recommendation Models in Federated Settings: A Survey](https://mlsanity.com/paper/2504.07101/) (2025)
- [A Multi-Expert Structural-Semantic Hybrid Framework for Unveiling Historical Patterns in Temporal Knowledge Graphs](https://mlsanity.com/paper/2506.14235/) (2025)
- [Multi-modal Adaptive Mixture of Experts for Cold-start Recommendation](https://mlsanity.com/paper/2508.08042/) (2025)
- [Catalog-Native LLM: Speaking Item-ID Dialect with Less Entanglement for Recommendation](https://mlsanity.com/paper/2510.05125/) (2025)
- [Synergy over Discrepancy: A Partition-Based Approach to Multi-Domain LLM Fine-Tuning](https://mlsanity.com/paper/2511.07198/) (2025)
- [LLM-EDT: Large Language Model Enhanced Cross-domain Sequential Recommendation with Dual-phase Training](https://mlsanity.com/paper/2511.19931/) (2025)
- [SemaCDR: LLM-Powered Transferable Semantics for Cross-Domain Sequential Recommendation](https://mlsanity.com/paper/2604.09551/) (2026)
- [Long-term User Engagement Optimization through Model-agnostic Downstream Rewards Learning](https://mlsanity.com/paper/2607.14192/) (2026)
- [Sharpness-aware Model Merging with Salience Recovery for LLM-based Cross-Domain Sequential Recommendation](https://mlsanity.com/paper/2607.25366/) (2026)
- [RouterKT: Mixture-of-Experts for Knowledge Tracing](https://mlsanity.com/paper/2504.08989/) (2025)
- [Empowering Denoising Sequential Recommendation with Large Language Model Embeddings](https://mlsanity.com/paper/2510.04239/) (2025)
- [No One Left Behind: How to Exploit the Incomplete and Skewed Multi-Label Data for Conversion Rate Prediction](https://mlsanity.com/paper/2512.13300/) (2025)
- [Distillation-based Scenario-Adaptive Mixture-of-Experts for the Matching Stage of Multi-scenario Recommendation](https://mlsanity.com/paper/2601.02368/) (2025)
- [CAMMSR: Category-Guided Attentive Mixture of Experts for Multimodal Sequential Recommendation](https://mlsanity.com/paper/2603.04320/) (2026)
- [Unbiased Rectification for Sequential Recommender Systems Under Fake Orders](https://mlsanity.com/paper/2604.08550/) (2026)
- [Action-Aware Generative Sequence Modeling for Short Video Recommendation](https://mlsanity.com/paper/2604.25834/) (2026)

## Abstract (arXiv)

Multi-domain recommendation and multi-task recommendation have demonstrated their effectiveness in leveraging common information from different domains and objectives for comprehensive user modeling. Nonetheless, the practical recommendation usually faces multiple domains and tasks simultaneously, which cannot be well-addressed by current methods. To this end, we introduce M3oE, an adaptive Multi-domain Multi-task Mixture-of-Experts recommendation framework. M3oE integrates multi-domain information, maps knowledge across domains and tasks, and optimizes multiple objectives. We leverage three mixture-of-experts modules to learn common, domain-aspect, and task-aspect user preferences respectively to address the complex dependencies among multiple domains and tasks in a disentangled manner. Additionally, we design a two-level fusion mechanism for precise control over feature extraction and fusion across diverse domains and tasks. The framework's adaptability is further enhanced by applying AutoML technique, which allows dynamic structure optimization. To the best of the authors' knowledge, our M3oE is the first effort to solve multi-domain multi-task recommendation self-adaptively. Extensive experiments on two benchmark datasets against diverse baselines demonstrate M3oE's superior performance. The implementation code is available to ensure reproducibility.