# MCMoE: Completing Missing Modalities with Mixture of Experts for Incomplete Multimodal Action Quality Assessment

arXiv: https://arxiv.org/abs/2511.17397 · HTML: https://mlsanity.com/paper/2511.17397/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts.
- Builds on 21 of its 54 references that are in the map.
- First posted on arXiv 21 Nov 2025.

## Placed in

- Mixture-of-experts / [Multimodal MoE models](https://mlsanity.com/moe/multimodal-moe-models/)

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [GAIN: Missing Data Imputation using Generative Adversarial Nets](https://mlsanity.com/paper/1806.02920/) (2018)
- [Multimodal Generative Models for Scalable Weakly-Supervised Learning](https://mlsanity.com/paper/1802.05335/) (2018)
- [Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models](https://mlsanity.com/paper/1911.03393/) (2019)
- [FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessment](https://mlsanity.com/paper/2204.03646/) (2022)
- [GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation](https://mlsanity.com/paper/2203.02177/) (2022)
- [Learning to score the figure skating sports videos](https://mlsanity.com/paper/1802.02774/) (2018)
- [Towards Good Practices for Missing Modality Robust Action Recognition](https://mlsanity.com/paper/2211.13916/) (2022)
- [MMANet: Margin-aware Distillation and Modality-aware Regularization for Incomplete Multimodal Learning](https://mlsanity.com/paper/2304.08028/) (2023)
- [TSA-Net: Tube Self-Attention Network for Action Quality Assessment](https://mlsanity.com/paper/2201.03746/) (2022)
- [FineParser: A Fine-grained Spatio-temporal Action Parser for Human-centric Action Quality Assessment](https://mlsanity.com/paper/2405.06887/) (2024)
- [Privacy Enhanced Multimodal Neural Representations for Emotion Recognition](https://mlsanity.com/paper/1910.13212/) (2019)
- [Skating-Mixer: Long-Term Sport Audio-Visual Modeling with MLPs](https://mlsanity.com/paper/2203.03990/) (2022)
- [Narrative Action Evaluation with Prompt-Guided Multimodal Interaction](https://mlsanity.com/paper/2404.14471/) (2024)
- [CoFInAl: Enhancing Action Quality Assessment with Coarse-to-Fine Instruction Alignment](https://mlsanity.com/paper/2404.13999/) (2024)
- [PASSION: Towards Effective Incomplete Multi-Modal Medical Image Segmentation with Imbalanced Missing Rates](https://mlsanity.com/paper/2407.14796/) (2024)
- [SDR-GNN: Spectral Domain Reconstruction Graph Neural Network for Incomplete Multimodal Learning in Conversational Emotion Recognition](https://mlsanity.com/paper/2411.19822/) (2024)
- [Do As I Do: Pose Guided Human Motion Copy](https://mlsanity.com/paper/2406.16601/) (2024)
- [Joint-Motion Mutual Learning for Pose Estimation in Videos](https://mlsanity.com/paper/2408.02285/) (2024)

## Built on by (most cited first, 7 of 7)

- [Enhancing Foundation VLM Robustness to Missing Modality: Scalable Diffusion for Bi-directional Feature Restoration](https://mlsanity.com/paper/2602.03151/) (2026)
- [MLCR: Multi-Level Cue Refinement for Long-Term Multimodal Action Quality Assessment](https://mlsanity.com/paper/2605.08945/) (2026)
- [GMENet: Generative Mixture of Experts Network for Multi-Center Glioma Diagnosis with Incomplete Imaging Sequences](https://mlsanity.com/paper/2605.23183/) (2026)
- [Context-driven Missing-Modality Learning for Robust Medical Diagnosis with Image-Tabular Data](https://mlsanity.com/paper/2605.25968/) (2026)
- [Explainable Forensics of Manipulated Segments in Untrimmed Long Videos](https://mlsanity.com/paper/2606.02402/) (2026)
- [MoAKE: Toward Unified All-in-One Action Quality Assessment via Mixture of Action Knowledge Experts](https://mlsanity.com/paper/2607.19826/) (2026)
- [Compass: Degradation-Simulated Reciprocal Learning with Lightweight Needle RWKV for Multimodal Crack Segmentation under Missing Modalities](https://mlsanity.com/paper/2608.03559/) (2026)

## Abstract (arXiv)

Multimodal Action Quality Assessment (AQA) has recently emerged as a promising paradigm. By leveraging complementary information across shared contextual cues, it enhances the discriminative evaluation of subtle intra-class variations in highly similar action sequences. However, partial modalities are frequently unavailable at the inference stage in reality. The absence of any modality often renders existing multimodal models inoperable. Furthermore, it triggers catastrophic performance degradation due to interruptions in cross-modal interactions. To address this issue, we propose a novel Missing Completion Framework with Mixture of Experts (MCMoE) that unifies unimodal and joint representation learning in single-stage training. Specifically, we propose an adaptive gated modality generator that dynamically fuses available information to reconstruct missing modalities. We then design modality experts to learn unimodal knowledge and dynamically mix the knowledge of all experts to extract cross-modal joint representations. With a mixture of experts, missing modalities are further refined and complemented. Finally, in the training phase, we mine the complete multimodal features and unimodal expert knowledge to guide modality generation and generation-based joint representation extraction. Extensive experiments demonstrate that our MCMoE achieves state-of-the-art results in both complete and incomplete multimodal learning on three public AQA benchmarks. Code is available at https://github.com/XuHuangbiao/MCMoE.