mlsanityData to 14 Oct 2026

MedMix: Specialization-Consistent Federated Sparse MoEs under Modality Heterogeneity

arXiv 2608.13911 ↗First posted 14 Aug 2026cs.LGcited by 0

On the map: Mixture-of-experts → Medical and biomedical MoE

Abstract (arXiv)

Federated multimodal medical AI faces modality heterogeneity at both the client and sample levels: clients may systematically lack access to specific modality types, while individual records within the same client may contain different partial modality subsets. Sparse Mixture-of-Experts (MoE) architectures are a promising remedy for modality-adaptive computation, but their use in federated learning is fragile under cross-client modality heterogeneity, where locally learned routing policies can diverge across clients and drive experts toward incompatible specializations. Different clients may assign the same observed modality configuration to different experts, or train similarly indexed experts on different missing-modality configurations, causing standard aggregation to misalign or overwrite the expert specialization that sparse MoEs are intended to learn. To address this challenge, we propose MedMix, a semantic-alignment framework for federated multimodal sparse MoEs that coordinates cross-client routing and expert specialization using modality context. At the client side, MedMix uses modality-context-aware routing to guide expert selection using each token's modality identity, position, and incompleteness context. Across clients, it uses consensus-guided routing alignment to construct server-side consensus anchors for shared modality patterns and align local routing distributions across clients. Complementing these routing mechanisms, client-adaptive expert aggregation leverages client-specific modality-pattern prototypes to match and aggregate functionally similar experts across clients. Experiments on real-world multimodal medical datasets show that MedMix achieves the best average F1 across diverse modality heterogeneity and modality incompleteness settings, with especially clear gains under severe heterogeneity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical and biomedical MoE, one of 12 directions in Mixture-of-experts. Builds on 21 of its 30 references that are in the map. First posted on arXiv 14 Aug 2026.

Builds on

21 of 30 references.

References · 21

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  7. 2019Advances and Open Problems in Federated Learningcited by 3,046
  8. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  9. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  10. 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 245
  11. 2016HeMIS: Hetero-Modal Image Segmentationcited by 103
  12. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 92
9 more
  1. 2024Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Expertscited by 70
  2. 2024FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusioncited by 68
  3. 2023FedMultimodal: A Benchmark For Multimodal Federated Learningcited by 51
  4. 2021Federated Mixture of Expertscited by 28
  5. 2024FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Expertscited by 27
  6. 2024FedMAC: Tackling Partial-Modality Missing in Federated Learning with Cross-Modal Aggregation and Contrastive Regularizationcited by 16
  7. 2025Learning Reconfigurable Representations for Multimodal Federated Learning with Missing Datacited by 8
  8. 2026Aggregation Alignment for Federated Learning with Mixture-of-Experts under Data Heterogeneitycited by 4
  9. 2024Federated Learning for Time-Series Healthcare Sensing with Incomplete Modalitiescited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026FM2: Unified Federated Foundation Models for Heterogeneous Multimodal Medical Imaging
  2. 2026ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
  3. 2024Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models
  4. 2026Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptation
  5. 2026Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learning
  6. 2026HFedMoE: Resource-aware Heterogeneous Federated Learning with Mixture-of-Experts
  7. 2024FedMoE-DA: Federated Mixture of Experts via Domain Aware Fine-grained Aggregation
  8. 2026SMoES: Soft Modality-Guided Expert Specialization in MoE-VLMs