mlsanityData to 14 Oct 2026

Role-Guided MOE for Encoder-Level Pathology Representation Learning in WSI Classification

arXiv 2609.34897 ↗First posted 28 Sep 2026cs.CVcited by 0

On the map: Mixture-of-experts → Medical and biomedical MoE

Abstract (arXiv)

Whole slide image classification is a fundamental task in computational pathology, where patch representation quality directly affects downstream aggregation and slide-level discriminability. Pathology foundation models are widely adopted as frozen feature extractors for WSI classification; however, their fixed encoders may produce representations insufficiently adapted to target-specific tissue patterns and discriminative cues. Fine-tuning can improve target adaptation, but introduces a trade-off between pathology-specific representation capacity and adaptation efficiency, particularly in data-scarce settings. To address this, we propose a pathology role-guided mixture-of-experts feed-forward network (MoE-FFN) framework for efficient encoder-level representation learning. We design a two-stage training paradigm to establish and adapt pathology-aware expert specialization. In source-domain expert initialization, pathology-specific priors are distilled from a frozen Virchow2 teacher into a lightweight DINOv2-small student, while role prototypes serve as weak pathological anchors to encourage distinct expert functions. MoE-FFN blocks are introduced into selected high-level transformer layers to provide transformation diversity for heterogeneous pathological patterns. In target-domain adaptation, the initialized experts are refined through asymmetric prototype-guided optimization, enhancing task-relevant positive evidence and separating confusable hard negatives. The resulting encoder extracts offline patch representations that can be directly integrated with standard MIL aggregators. Experiments on the public BRACS dataset and a private PAROTID WSI dataset across five representative backbones demonstrate consistent improvements over the strongest baseline.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical and biomedical MoE, one of 12 directions in Mixture-of-experts. Builds on 24 of its 36 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

24 of 36 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024M4: Multi-Proxy Multi-Gate Mixture of Experts Network for Multiple Instance Learning in Histopathology Image Analysis24 Jul

Other references · 23

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  6. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  7. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  8. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  9. 2018Attention-based Deep Multiple Instance Learningcited by 1,095
  10. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  11. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  12. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 745
11 more
  1. 2021TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classificationcited by 555
  2. 2024Multimodal Whole Slide Foundation Model for Pathologycited by 162
  3. 2022On the Representation Collapse of Sparse Mixture of Expertscited by 157
  4. 2022Benchmarking Self-Supervised Learning on Diverse Pathology Datasetscited by 150
  5. 2021BRACS: A Dataset for BReAst Carcinoma Subtyping in H&E Histology Imagescited by 132
  6. 2024Feature Re-Embedding: Towards Foundation Model-Level Performance in Computational Pathologycited by 67
  7. 2024Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer Modelscited by 56
  8. 2024Benchmarking Pathology Foundation Models: Adaptation Strategies and Scenarioscited by 17
  9. 2025SPIDER: A Comprehensive Multi-Organ Supervised Pathology Dataset and Baseline Modelscited by 13
  10. 2024Free Lunch in Pathology Foundation Model: Task-specific Model Adaptation with Concept-Guided Feature Enhancementcited by 11
  11. 2024PathoTune: Adapting Visual Foundation Model to Pathological Specialistscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024A Mixture of Experts (MoE) model to improve AI-based computational pathology prediction performance under variable levels of histopathology image blur
  2. 2022Self-Supervised Vision Transformers Learn Visual Concepts in Histopathology
  3. 2025Slide-Level Prompt Learning with Vision Language Models for Few-Shot Multiple Instance Learning in Histopathology
  4. 2026LaGuadia: Language-Guided Adaptive Distillation from Pathology Foundation Models
  5. 2026Unified Multi-Foundation-Model Slide Representation for Pan-Cancer Recognition and Text-Guided Tumor Localization
  6. 2026Multimodal Model for Computational Pathology:Representation Learning and Image Compression
  7. 2024Unsupervised Foundation Model-Agnostic Slide-Level Representation Learning
  8. 2025Pathryoshka: Compressing Pathology Foundation Models via Multi-Teacher Knowledge Distillation with Nested Embeddings