mlsanityData to 14 Oct 2026

Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey

arXiv 2605.27431 ↗First posted 22 May 2026cs.LG cs.AIcited by 0

On the map: Mixture-of-experts → Multimodal MoE models

Abstract (arXiv)

Mixture-of-Experts (MoE) presents a naturally compatible and scalable framework for multimodal learning, demonstrating strong adaptability across diverse modalities and tasks. Despite its growing success, a comprehensive and systematic review on the MoE metho addressing multimodal challenges remains lacking. Existing surveys tend to evaluate either multimodal learning or MoE independently from method taxonomy, overlooking the unique interplay between them. This survey fills that gap by answering a central question: textitHow does MoE effectively resolve multimodal challenges? We approach this from three key perspectives: (1) MoE as an Efficient Multimodal Engine: enabling scalable multimodal modeling by decoupling computational cost from parameter growth and mitigating modality redundancy through selective expert activation; (2) MoE as a Multimodal Representation Learner: integrating complementary multi-opinion expert knowledge to enrich alignment and interaction representations; and (3) MoE as a Multimodal Adapter: providing a modular and flexible mechanism to model imperfect data scenarios such as modality imbalance and missing modality. Through our extensive literature review, we identify critical research gaps, including interpretable routing, expert communication, modality integration, and lifelong multimodal learning. We position this survey as a foundation for future research toward interpretable and sustainable multimodal Mixture-of-Experts system.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 42 of its 49 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

42 of 49 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025I2MoE: Interpretable Multimodal Interaction-aware Mixture-of-Experts25 May
  2. 2025CL-MoE: Enhancing Multimodal Large Language Model with Dual Momentum Mixture-of-Experts for Continual Visual Question Answering1 Mar
  3. 2025R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts27 Feb
  4. 2024Lifelong Knowledge Editing for Vision Language Models with Low-Rank Mixture-of-Experts23 Nov
  5. 2024Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts10 Oct
  6. 2024MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts31 Jul
  7. 2024Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts18 May
  8. 2024CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts9 May
  9. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Models29 Jan
  10. 2023Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts1 Dec
  11. 2023Alternating Gradient Descent and Mixture-of-Experts for Integrated Multimodal Perception10 May
  12. 2022Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts6 Jun

Other references · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  3. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 459
  4. 2022Multimodal Learning with Transformers: A Surveycited by 400
  5. 2019What Makes Training Multi-Modal Classification Networks Hard?cited by 344
  6. 2023From Sparse to Soft Mixtures of Expertscited by 248
  7. 2024A Survey on Mixture of Experts in Large Language Modelscited by 237
  8. 2024Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapterscited by 199
  9. 2023Multimodal Large Language Models: A Surveycited by 195
  10. 2024Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Modelscited by 146
  11. 2023RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Pathscited by 142
  12. 2024Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoderscited by 139
18 more
  1. 2022ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Expertscited by 112
  2. 2024Efficient Multimodal Large Language Models: A Surveycited by 96
  3. 2024MoVA: Adapting Mixture of Vision Experts to Multimodal Contextcited by 83
  4. 2024FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusioncited by 68
  5. 2024Upcycling Large Language Models into Mixture of Expertscited by 49
  6. 2024Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Modelscited by 46
  7. 2024MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Modelscited by 46
  8. 2024MoAI: Mixture of All Intelligence for Large Language and Vision Modelscited by 36
  9. 2024M4oE: A Foundation Model for Medical Multimodal Image Segmentation with Mixture of Expertscited by 26
  10. 2023MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Expertscited by 21
  11. 2024SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Expertscited by 16
  12. 2024γ-MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Modelscited by 14
  13. 2024Multiple Heads are Better than One: Mixture of Modality Knowledge Experts for Entity Representation Learningcited by 13
  14. 2024From Efficient Multimodal Models to World Models: A Surveycited by 13
  15. 2025MedMoE: Modality-Specialized Mixture of Experts for Medical Vision-Language Understandingcited by 10
  16. 2026PrivGemo: Privacy-Preserving Dual-Tower Graph Retrieval for Empowering LLM Reasoning with Memory Augmentationcited by 9
  17. 2025UniMoD: Efficient Unified Multimodal Transformers with Mixture-of-Depthscited by 4
  18. 2025Rethinking Gating Mechanism in Sparse MoE: Handling Arbitrary Modality Inputs with Confidence-Guided Gatecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptation
  2. 2026Symbiotic-MoE: Unlocking the Synergy between Generation and Understanding
  3. 2026FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learning
  4. 2026Toward Structural Multimodal Representations: Specialization, Selection, and Sparsification via Mixture-of-Experts
  5. 2026Feature-level Interaction Explanations in Multimodal Transformers
  6. 2026MoST: Mixing Speech and Text with Modality-Aware Mixture of Experts
  7. 2026MoEMB: Scaling Universal Multimodal Embeddings with Efficient Mixture-of-Experts Models
  8. 2026Scaling Laws for Multimodal Data Mixtures