# Multimodal MoE models

Area: [Mixture-of-experts](https://mlsanity.com/moe/index.md). HTML: https://mlsanity.com/moe/multimodal-moe-models/

## Key facts

- Multimodal MoE models: 221 papers on arXiv since 2017 (data through 7 Oct 2026).
- 25 of its 221 mapped papers (11%) cite “MoE-LLaVA: Mixture of Experts for Large Vision-Language Models” (2024, arXiv 2401.15947).
- 17 of its 221 mapped papers (8%) cite “Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts” (2022, arXiv 2206.02770).
- 8 new papers 17 Sep–7 Oct.

## Most cited here

- [MoE-LLaVA: Mixture of Experts for Large Vision-Language Models](https://mlsanity.com/paper/2401.15947/) (2024), cited by 25 papers here
- [Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Experts](https://mlsanity.com/paper/2206.02770/) (2022), cited by 17 papers here
- [SpeechMoE: Scaling to Large Acoustic Models with Dynamic Routing Mixture of Experts](https://mlsanity.com/paper/2105.03036/) (2021), cited by 13 papers here
- [Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts](https://mlsanity.com/paper/2405.11273/) (2024), cited by 13 papers here
- [MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts](https://mlsanity.com/paper/2407.21770/) (2024), cited by 12 papers here
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024), cited by 12 papers here
- [CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts](https://mlsanity.com/paper/2405.05949/) (2024), cited by 11 papers here
- [Mixture-of-Expert Conformer for Streaming Multilingual ASR](https://mlsanity.com/paper/2305.15663/) (2023), cited by 9 papers here
- [Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognition](https://mlsanity.com/paper/2307.05956/) (2023), cited by 8 papers here
- [Aria: An Open Multimodal Native Mixture-of-Experts Model](https://mlsanity.com/paper/2410.05993/) (2024), cited by 7 papers here

## New (17 Sep–7 Oct)

- [SEAL: Mixture-Closed Additive Reconstruction and Refinement-Aware Expert Routing for Efficient Speech Separation](https://mlsanity.com/paper/2610.07047/) (2026-10-05)
- [UniAE-MoE: A Unified Audio Encoder via Mixture of Experts](https://mlsanity.com/paper/2609.39199/) (2026-09-30)
- [Domain-Adaptive Dual-Gating Mixture of Experts for Generalizable Speech Deepfake Detection](https://mlsanity.com/paper/2609.33709/) (2026-09-27)
- [OneSign: Unifying Sign Language Understanding Tasks with One Model](https://mlsanity.com/paper/2609.33090/) (2026-09-27)
- [OmniMoE-VL: A Sparse Vision-Language Model with Coupled Visual-Depth Routing](https://mlsanity.com/paper/2609.32780/) (2026-09-26)
- [Qwen3.8-Omni: Towards Native Omni-Modal Agents](https://mlsanity.com/paper/2609.25611/) (2026-09-22)
- [MECT: Mixture of Experts with CNN-Transformer Network for Speaker verification](https://mlsanity.com/paper/2609.24061/) (2026-09-21)
- [All-in-One Multilingual Scene Text Recognition with Script-aware Mixture-of-Experts](https://mlsanity.com/paper/2609.24058/) (2026-09-21)