# Multimodal Distillation-Driven Ensemble Learning for Long-Tailed Histopathology Whole Slide Images Analysis

arXiv: https://arxiv.org/abs/2503.00915 · HTML: https://mlsanity.com/paper/2503.00915/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 28 references that are in the map.
- First posted on arXiv 2 Mar 2025.

## Builds on (most cited first, 15 of 15)

- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [BBN: Bilateral-Branch Network with Cumulative Learning for Long-Tailed Visual Recognition](https://mlsanity.com/paper/1912.02413/) (2019)
- [DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification](https://mlsanity.com/paper/2203.12081/) (2022)
- [Dynamic Graph Representation with Knowledge-aware Attention for Histopathology Whole Slide Image Analysis](https://mlsanity.com/paper/2403.07719/) (2024)
- [Additive MIL: Intrinsically Interpretable Multiple Instance Learning for Pathology](https://mlsanity.com/paper/2206.01794/) (2022)
- [Agent Aggregator with Mask Denoise Mechanism for Histopathology Whole Slide Image Analysis](https://mlsanity.com/paper/2409.11664/) (2024)
- [MergeUp-augmented Semi-Weakly Supervised Learning for WSI Classification](https://mlsanity.com/paper/2408.12825/) (2024)
- [VLM-KD: Knowledge Distillation from VLM for Long-Tail Visual Recognition](https://mlsanity.com/paper/2408.16930/) (2024)
- [Diagnostic Text-guided Representation Learning in Hierarchical Classification for Pathological Whole Slide Image](https://mlsanity.com/paper/2411.10709/) (2024)
- [Towards a Comprehensive Benchmark for Pathological Lymph Node Metastasis in Breast Cancer Sections](https://mlsanity.com/paper/2411.10752/) (2024)
- [Dynamic Hypergraph Representation for Bone Metastasis Cancer Analysis](https://mlsanity.com/paper/2501.16787/) (2025)

## Built on by (most cited first, 1 of 1)

- [DeCo-MIL: Debiased Counterfactual Reasoning for Long-Tailed Whole Slide Image Analysis](https://mlsanity.com/paper/2608.14719/) (2026)

## Abstract (arXiv)

Multiple Instance Learning (MIL) plays a significant role in computational pathology, enabling weakly supervised analysis of Whole Slide Image (WSI) datasets. The field of WSI analysis is confronted with a severe long-tailed distribution problem, which significantly impacts the performance of classifiers. Long-tailed distributions lead to class imbalance, where some classes have sparse samples while others are abundant, making it difficult for classifiers to accurately identify minority class samples. To address this issue, we propose an ensemble learning method based on MIL, which employs expert decoders with shared aggregators and consistency constraints to learn diverse distributions and reduce the impact of class imbalance on classifier performance. Moreover, we introduce a multimodal distillation framework that leverages text encoders pre-trained on pathology-text pairs to distill knowledge and guide the MIL aggregator in capturing stronger semantic features relevant to class information. To ensure flexibility, we use learnable prompts to guide the distillation process of the pre-trained text encoder, avoiding limitations imposed by specific prompts. Our method, MDE-MIL, integrates multiple expert branches focusing on specific data distributions to address long-tailed issues. Consistency control ensures generalization across classes. Multimodal distillation enhances feature extraction. Experiments on Camelyon+-LT and PANDA-LT datasets show it outperforms state-of-the-art methods.