mlsanityData to 7 Oct 2026

Multimodal MoE models

221 papers1 this weekMedium confidence in this grouping: mostly one topic, with blurry edgesMedium confidence in this grouping: mostly one topic, with blurry edges · About

Keywords: multimodal, speech, experts, visual, moe

Key facts. Multimodal MoE models: 221 papers on arXiv since 2017 (data through 7 Oct 2026). 25 of its 221 mapped papers (11%) cite “MoE-LLaVA: Mixture of Experts for Large Vision-Language Models” (2024, arXiv 2401.15947). 17 of its 221 mapped papers (8%) cite “Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts” (2022, arXiv 2206.02770). 8 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

8 new papers, 17 Sep – 7 Oct; 1 this week.

  1. This week · 1
  2. SEAL: Mixture-Closed Additive Reconstruction and Refinement-Aware Expert Routing for Efficient Speech Separation also fits: Expert routing strategies
  3. Earlier · 7
  4. UniAE-MoE: A Unified Audio Encoder via Mixture of Experts
  5. Domain-Adaptive Dual-Gating Mixture of Experts for Generalizable Speech Deepfake Detection
  6. OneSign: Unifying Sign Language Understanding Tasks with One Model
  7. OmniMoE-VL: A Sparse Vision-Language Model with Coupled Visual-Depth Routing also fits: Vision mixture-of-experts
  8. Qwen3.8-Omni: Towards Native Omni-Modal Agents also fits: MoE in reinforcement learning and robotics
  9. MECT: Mixture of Experts with CNN-Transformer Network for Speaker verification
  10. All-in-One Multilingual Scene Text Recognition with Script-aware Mixture-of-Experts

Foundations

Most cited by papers here

  1. MoE-LLaVA: Mixture of Experts for Large Vision-Language Models2024cited by 25 here
  2. Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts2022cited by 17 here
  3. SpeechMoE: Scaling to Large Acoustic Models with Dynamic Routing Mixture of Experts2021cited by 13 here
  4. Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts2024cited by 13 here
  5. MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts2024cited by 12 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 72 here
  2. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 36 here
  3. 2021Scaling Vision with Sparse Mixture of Expertscited by 16 here
  4. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 17 here
  5. 2022Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Expertscited by 17 here
  6. 2022Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpointscited by 17 here
  7. 2023Scaling Vision-Language Models with Sparse Mixture of Expertscited by 14 here
  8. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 25 here
  9. 2024Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Expertscited by 13 here
15 more it builds on, not in this chain
  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 63 here
  2. 2021SpeechMoE: Scaling to Large Acoustic Models with Dynamic Routing Mixture of Expertscited by 13 here
  3. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 14 here
  4. 2022DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scalecited by 8 here
  5. 2022Mixture-of-Experts with Expert Choice Routingcited by 11 here
  6. 2023Mixture-of-Expert Conformer for Streaming Multilingual ASRcited by 9 here
  7. 2023Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognitioncited by 8 here
  8. 2024Mixtral of Expertscited by 12 here
  9. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 19 here
  10. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 7 here
  11. 2024CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Expertscited by 11 here
  12. 2024MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Expertscited by 12 here
  13. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 12 here
  14. 2024DeepSeek-V3 Technical Reportcited by 9 here
  15. 2025Qwen3 Technical Reportcited by 10 here

The 24 most cited of 280.

Browse

30 of 221, ranked 6–35

  1. 6DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding202412
  2. 7CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts202411
  3. 8Mixture-of-Expert Conformer for Streaming Multilingual ASR20239
  4. 9Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognition20238
  5. 10Aria: An Open Multimodal Native Mixture-of-Experts Model20247
  6. 11Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts20246
  7. 12BA-MoE: Boundary-Aware Mixture-of-Experts Adapter for Code-Switching Speech Recognition20236
  8. 13SpeechMoE2: Mixture-of-Experts Model with Improved Routing20216
  9. 14Kimi-VL Technical Report20255
  10. 15EVE: Efficient Vision-Language Pre-training with Masked Prediction and Modality-Aware MoE20235
All 30
  1. 16R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts20253
  2. 17CL-MoE: Enhancing Multimodal Large Language Model with Dual Momentum Mixture-of-Experts for Continual Visual Question Answering20253
  3. 18Enhancing Multi-modal Models with Heterogeneous MoE Adapters for Fine-tuning20253
  4. 19I2MoE: Interpretable Multimodal Interaction-aware Mixture-of-Experts20253
  5. 20MoIIE: Mixture of Intra- and Inter-Modality Experts for Large Vision Language Models20253
  6. 21Qwen3-VL Technical Report20253
  7. 22MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training20243
  8. 23SC-MoE: Switch Conformer Mixture of Experts for Unified Streaming and Non-streaming Code-Switching ASR20243
  9. 24Boosting Code-Switching ASR with Mixture of Experts Enhanced Speech-Conditioned LLM20243
  10. 25HDMoLE: Mixture of LoRA Experts with Hierarchical Routing and Dynamic Thresholds for Fine-Tuning LLM-based ASR Models20243
  11. 26MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning20243
  12. 27Seed1.5-VL Technical Report20252
  13. 28MoDES: Accelerating Mixture-of-Experts Multimodal Large Language Models via Dynamic Expert Skipping20252
  14. 29An Effective Mixture-Of-Experts Approach For Code-Switching Speech Recognition Leveraging Encoder Disentanglement20242
  15. 30U2++ MoE: Scaling 4.7x parameters with minimal impact on RTF20242
  16. 31SAML: Speaker Adaptive Mixture of LoRA Experts for End-to-End ASR20242
  17. 32Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.020242
  18. 33Leveraging Mixture of Experts for Improved Speech Deepfake Detection20242
  19. 34CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling20242
  20. 35Alternating Gradient Descent and Mixture-of-Experts for Integrated Multimodal Perception20232