mlsanityData to 14 Oct 2026

MACS: Modality-Aware Capacity Scaling for Efficient Multimodal MoE Inference

arXiv 2605.05225 ↗First posted 19 Apr 2026cs.LG cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

also in 1 more
Abstract (arXiv)

Mixture-of-Experts Multimodal Large Language Models (MoE MLLMs) suffer from a significant efficiency bottleneck during Expert Parallelism (EP) inference due to the straggler effect. This issue is worsened in the multimodal context, as existing token-count-based load balancing methods fail to address two unique challenges: (1) Information Heterogeneity, where numerous redundant visual tokens are treated equally to semantically critical ones, and (2) Modality Dynamics, where varying visual to text ratios across tasks lead to resource misallocation. To address these challenges, we propose MACS (Modality-Aware Capacity Scaling), a training-free inference framework. Specifically, MACS introduces an Entropy-Weighted Load mechanism to quantify the semantic value of visual tokens, addressing information heterogeneity. Additionally, the Dynamic Modality-Adaptive Capacity mechanism allocates expert resources based on the real-time modal composition of the input. Extensive experiments demonstrate that MACS significantly outperforms existing methods on various multimodal benchmarks, providing a novel and robust solution for the efficient deployment of MoE MLLMs in EP inference.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 24 of its 35 references that are in the map. First posted on arXiv 19 Apr 2026.

Builds on

24 of 35 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024LLaVA-OneVision: Easy Visual Task Transfer6 Aug
  2. 2023Improved Baselines with Visual Instruction Tuning5 Oct
  3. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities4 Aug
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul

Other references · 20

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  3. 2019Towards VQA Models That Can Readcited by 1,718
  4. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  5. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  6. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,413
  7. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,109
  8. 2023EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understandingcited by 680
  9. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 665
  10. 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 236
  11. 2024OpenMoE: An Early Effort on Open Mixture-of-Experts Language Modelscited by 187
  12. 2024MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Routercited by 62
8 more
  1. 2024Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Expertscited by 32
  2. 2025Token Activation Map to Visually Explain Multimodal LLMscited by 32
  3. 2024STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruningcited by 30
  4. 2024LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Trainingcited by 26
  5. 2025Capacity-Aware Inference: Mitigating the Straggler Effect in Mixture of Expertscited by 15
  6. 2024MIT-10M: A Large Scale Parallel Corpus of Multilingual Image Translationcited by 12
  7. 2025Merge then Realign: Simple and Effective Modality-Incremental Continual Learning for Multimodal LLMscited by 6
  8. 2025Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Modelscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026DecoMoE: Decoupling Visual Propagation and Expert Computation for Efficient Multimodal MoE Inferencecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts
  2. 2025Modality Inflation: Energy Characterization and Optimization Opportunities for MLLM Inference ↗
  3. 2026ReaLB: Real-Time Load Balancing for Multimodal MoE Inference
  4. 2025MoDES: Accelerating Mixture-of-Experts Multimodal Large Language Models via Dynamic Expert Skipping
  5. 2025ElasticMM: Efficient Multimodal LLMs Serving with Elastic Multimodal Parallelism
  6. 2026Scaling Laws for Multimodal Data Mixtures
  7. 2025Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inference
  8. 2024Accelerating Multimodal Large Language Models via Dynamic Visual-Token Exit and the Empirical Findings