# Scale Federated Learning for Label Set Mismatch in Medical Image Classification

arXiv: https://arxiv.org/abs/2304.06931 · HTML: https://mlsanity.com/paper/2304.06931/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 23 references that are in the map.
- First posted on arXiv 14 Apr 2023.

## Builds on (most cited first, 17 of 17)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [FedBN: Federated Learning on Non-IID Features via Local Batch Normalization](https://mlsanity.com/paper/2102.07623/) (2021)
- [Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning](https://mlsanity.com/paper/2006.12097/) (2020)
- [HarmoFL: Harmonizing Local and Global Drifts in Federated Learning on Heterogeneous Medical Images](https://mlsanity.com/paper/2112.10775/) (2021)
- [Federated Semi-supervised Medical Image Classification via Inter-client Relation Matching](https://mlsanity.com/paper/2106.08600/) (2021)
- [Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation](https://mlsanity.com/paper/2203.10144/) (2022)
- [Deep Mining External Imperfect Data for Chest X-ray Disease Screening](https://mlsanity.com/paper/2006.03796/) (2020)
- [Dynamic Bank Learning for Semi-supervised Federated Image Diagnosis with Class Imbalance](https://mlsanity.com/paper/2206.13079/) (2022)
- [Federated Learning with Positive and Unlabeled Data](https://mlsanity.com/paper/2106.10904/) (2021)
- [OXnet: Omni-supervised Thoracic Disease Detection from Chest X-rays](https://mlsanity.com/paper/2104.03218/) (2021)

## Built on by (most cited first, 3 of 3)

- [Enable the Right to be Forgotten with Federated Client Unlearning in Medical Imaging](https://mlsanity.com/paper/2407.02356/) (2024)
- [Federated Learning with Heterogeneous and Private Label Sets](https://mlsanity.com/paper/2508.18774/) (2025)
- [Self-Supervised Federated Learning under Data Heterogeneity for Label-Scarce Diatom Classification](https://mlsanity.com/paper/2603.29633/) (2026)

## Abstract (arXiv)

Federated learning (FL) has been introduced to the healthcare domain as a decentralized learning paradigm that allows multiple parties to train a model collaboratively without privacy leakage. However, most previous studies have assumed that every client holds an identical label set. In reality, medical specialists tend to annotate only diseases within their area of expertise or interest. This implies that label sets in each client can be different and even disjoint. In this paper, we propose the framework FedLSM to solve the problem of Label Set Mismatch. FedLSM adopts different training strategies on data with different uncertainty levels to efficiently utilize unlabeled or partially labeled data as well as class-wise adaptive aggregation in the classification layer to avoid inaccurate aggregation when clients have missing labels. We evaluated FedLSM on two public real-world medical image datasets, including chest X-ray (CXR) diagnosis with 112,120 CXR images and skin lesion diagnosis with 10,015 dermoscopy images, and showed that it significantly outperformed other state-of-the-art FL algorithms. The code can be found at https://github.com/dzp2095/FedLSM.