mlsanityData to 14 Oct 2026

Generalist-Specialist Mixture-of-Experts for Rare Pathology Detection in Multimodal Imaging

arXiv 2609.18688 ↗First posted 16 Sep 2026cs.CV cs.AIcited by 0

On the map: Mixture-of-experts → Medical and biomedical MoE

Abstract (arXiv)

AI models for multimodal medical imaging must balance modality-specific specialization with cross-modal shared representations, a trade-off that pure Mixture-of-Experts (MoE) architectures currently fail to satisfy. Expert-based routing improves in-domain learning but may sacrifice cross-modal signals, which appear particularly important for rare (low-prevalence) pathologies in our experiments. To resolve this, we introduce Generalist-Specialist-MoE (GS-MoE), a two-branch (MoE) architecture that couples a cross-modal generalist model with distinct modality-specific specialists (experts) via domain-constrained feature fusion. On RadImageNet (1.35M images, 165 pathologies, three modalities), GS-MoE recovers detection of six low-prevalence pathologies on which every baseline scores F1 = 0, with per-class gains up to +0.60 F1. It attains this while even slightly exceeding dense and specialist-only MoE aggregate baselines (MCC 0.770), while using ~53% fewer active parameters at inference than the strongest investigated dense model.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical and biomedical MoE, one of 12 directions in Mixture-of-experts. Builds on 29 of its 36 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

29 of 36 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025MedMoE: Modality-Specialized Mixture of Experts for Medical Vision-Language Understanding10 Jun
  2. 2024M4: Multi-Proxy Multi-Gate Mixture of Experts Network for Multiple Instance Learning in Histopathology Image Analysis24 Jul
  3. 2024SAM-Med3D-MoE: Towards a Non-Forgetting Segment Anything Model via Mixture of Experts for 3D Medical Image Segmentation6 Jul
  4. 2024M4oE: A Foundation Model for Medical Multimodal Image Segmentation with Mixture of Experts15 May
  5. 2024Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Models16 Apr
  6. 2022Patcher: Patch Transformers with Mixture of Experts for Precise Medical Image Segmentation3 Jun

Other references · 23

  1. 2016Densely Connected Convolutional Networkscited by 6,720
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  4. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  5. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  6. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
  7. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  8. 2016Domain Separation Networkscited by 320
  9. 2024A Survey on Mixture of Experts in Large Language Modelscited by 237
  10. 2018Characterizing and Avoiding Negative Transfercited by 229
  11. 2019Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networkscited by 222
  12. 2022Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpointscited by 209
11 more
  1. 2022A Review of Sparse Expert Models in Deep Learningcited by 148
  2. 2024CheXpert Plus: Augmenting a Large Chest X-ray Dataset with Text Radiology Reports, Patient Demographics and Additional Image Formatscited by 106
  3. 2025A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applicationscited by 91
  4. 2023ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learningcited by 28
  5. 2023Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Expertscited by 26
  6. 2025Mixture of Experts (MoE): A Big Data Perspectivecited by 18
  7. 2024Low-Rank Mixture-of-Experts for Continual Medical Image Segmentationcited by 16
  8. 2024Equipping Computational Pathology Systems with Artifact Processing Pipelines: A Showcase for Computation and Performance Trade-offscited by 11
  9. 2024Mixture of Experts in Image Classification: What's the Sweet Spot?cited by 11
  10. 2022Mitigating Negative Transfer in Multi-Task Learning with Exponential Moving Average Loss Weighting Strategiescited by 10
  11. 2023How Can We Tame the Long-Tail of Chest X-ray Datasets?cited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024GSCo: Towards Generalizable AI in Medicine via Generalist-Specialist Collaboration
  2. 2026Hierarchical MoE for Multi-Modal ILD Diagnosis
  3. 2025REN: Anatomically-Informed Mixture-of-Experts for Interstitial Lung Disease Diagnosis
  4. 2026TIER-MoE: Trust-Informed Expert Routing via Conditional Modality Risk for Multimodal Fusion in Biomedical Classification
  5. 2026ConceptM3oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology
  6. 2025Generalist Models in Medical Image Segmentation: A Survey and Performance Comparison with Task-Specific Approaches
  7. 2024Generalizable Single-Source Cross-modality Medical Image Segmentation via Invariant Causal Mechanisms
  8. 2024Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts