mlsanityData to 7 Oct 2026

GeMoE: Gating Entropy is All You Need for Uncertainty-aware Adaptive Routing in MoE-based Large Vision-Language Models

arXiv 2606.26287 ↗First posted 24 Jun 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

also in 1 more
Abstract (arXiv)

With the increase in model parameters and training data, the instruction following and generalization capabilities of Large VisionLanguage Models (LVLMs) have been significantly improved. Based on the Mixture of Experts (MoE) architecture, LVLMs expand their parameter capacity while maintaining the inference cost. However, traditional MoE methods employ a Top-k static routing strategy, which fails to account for variations in the input and adaptively select the number of experts, resulting in suboptimal resource utilization. In this paper, we propose viewing token routing as an information encoding task, framing dynamic routing as a Minimum Description Length (MDL) problem in encoding By validating the connection between MDL and gating entropy in the MoE scenario, we introduce Gating Entropy-based Uncertainty-aware Adaptive Routing (GeMoE) for MoE. Unlike traditional static or heuristic-based dynamic routing methods, GeMoE explicitly models the trade-off between model complexity and performance. By using gating entropy to assess the complexity of tokens, GeMoE adaptively determines the number of experts each token should engage. On a wide range of backbones and benchmarks, our method achieves 99.5% average performance retention compared to the original static routing, while improving average expert activation sparsity by 36.5%.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 65 references that are in the map. First posted on arXiv 24 Jun 2026.

Builds on

34 of 65 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Qwen3-VL Technical Report26 Nov
  2. 2025Long-Tailed Distribution-Aware Router For Mixture-of-Experts in Large Vision-Language Model2 Jul
  3. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding13 Dec
  4. 2023Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuning19 Dec
  5. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities4 Aug

Other references · 29

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2025Qwen2.5-VL Technical Reportcited by 5,418
  6. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  7. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  8. 2023Qwen Technical Reportcited by 3,273
  9. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  10. 2024Mixtral of Expertscited by 1,785
  11. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  12. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
17 more
  1. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 439
  2. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 291
  3. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 272
  4. 2023From Sparse to Soft Mixtures of Expertscited by 245
  5. 2022Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpointscited by 207
  6. 2024OpenMoE: An Early Effort on Open Mixture-of-Experts Language Modelscited by 185
  7. 2023Fast Inference of Mixture-of-Experts Language Models with Offloadingcited by 88
  8. 2023Mixture-of-Experts Meets Instruction Tuning:A Winning Combination for Large Language Modelscited by 65
  9. 2024Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Modelscited by 57
  10. 2024Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Modelscited by 55
  11. 2024MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Expertscited by 55
  12. 2024ReMoE: Fully Differentiable Mixture-of-Experts with ReLU Routingcited by 53
  13. 2024AdaMoLE: Fine-Tuning Large Language Models with Adaptive Mixture of Low-Rank Adaptation Expertscited by 33
  14. 2023Octavius: Mitigating Task Interference in MLLMs via LoRA-MoEcited by 25
  15. 2025Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Datacited by 21
  16. 2024Exploring Training on Heterogeneous Data with Mixture of Low-rank Adapterscited by 12
  17. 2025BlockFFN: Towards End-Side Acceleration-Friendly Mixture-of-Experts with Chunk-Level Activation Sparsitycited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Input Domain Aware MoE: Decoupling Routing Decisions from Task Optimization in Mixture of Experts
  2. 2026Beyond Global Routing Aggregation: Phase-Aware Expert Merging for MoE Vision-Language Models
  3. 2026MoE-GRPO: Optimizing Mixture-of-Experts via Reinforcement Learning in Vision-Language Models
  4. 2024Routers in Vision Mixture of Experts: An Empirical Study
  5. 2024Routing Experts: Learning to Route Dynamic Experts in Multi-modal Large Language Models
  6. 2026On Token's Dilemma: Dynamic MoE with Drift-Aware Token Assignment for Continual Learning of Large Vision Language Models
  7. 2024LIBMoE: A Library for comprehensive benchmarking Mixture of Experts in Large Language Models
  8. 2025How Many Experts Are Enough? Towards Optimal Semantic Specialization for Mixture-of-Experts