# Federated Multi-organ Segmentation with Inconsistent Labels

arXiv: https://arxiv.org/abs/2206.07156 · HTML: https://mlsanity.com/paper/2206.07156/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 55 references that are in the map.
- First posted on arXiv 14 Jun 2022.

## Builds on (most cited first, 30 of 34)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [FedBN: Federated Learning on Non-IID Features via Local Batch Normalization](https://mlsanity.com/paper/2102.07623/) (2021)
- [Federated Learning Based on Dynamic Regularization](https://mlsanity.com/paper/2111.04263/) (2021)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space](https://mlsanity.com/paper/2103.06030/) (2021)
- [The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challenge](https://mlsanity.com/paper/1912.01054/) (2019)
- [The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes](https://mlsanity.com/paper/1904.00445/) (2019)
- [Machine Learning Models that Remember Too Much](https://mlsanity.com/paper/1709.07886/) (2017)
- [DoDNet: Learning to segment multi-organ and tumors from multiple partially labeled datasets](https://mlsanity.com/paper/2011.10217/) (2020)
- [Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond](https://mlsanity.com/paper/2102.01998/) (2021)
- [Multi-organ Segmentation over Partially Labeled Datasets with Multi-scale Feature Abstraction](https://mlsanity.com/paper/2001.00208/) (2020)
- [Federated Learning for Breast Density Classification: A Real-World Implementation](https://mlsanity.com/paper/2009.01871/) (2020)
- [Learning from Multiple Datasets with Heterogeneous and Partial Labels for Universal Lesion Detection in CT](https://mlsanity.com/paper/2009.02577/) (2020)
- [Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation](https://mlsanity.com/paper/2203.10144/) (2022)
- [Federated Semi-Supervised Learning for COVID Region Segmentation in Chest CT using Multi-National Data from China, Italy, Japan](https://mlsanity.com/paper/2011.11750/) (2020)
- [FedMix: Mixed Supervised Federated Learning for Medical Image Segmentation](https://mlsanity.com/paper/2205.01840/) (2022)
- [IOP-FL: Inside-Outside Personalization for Federated Medical Image Segmentation](https://mlsanity.com/paper/2204.08467/) (2022)
- [Auto-FedAvg: Learnable Federated Averaging for Multi-Institutional Medical Image Segmentation](https://mlsanity.com/paper/2104.10195/) (2021)
- [Data Harmonisation for Information Fusion in Digital Healthcare: A State-of-the-Art Systematic Review, Meta-Analysis and Future Research Directions](https://mlsanity.com/paper/2201.06505/) (2022)
- [Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentation](https://mlsanity.com/paper/2203.06338/) (2022)

## Built on by (most cited first, 15 of 15)

- [Federated Learning for Medical Image Analysis: A Survey](https://mlsanity.com/paper/2306.05980/) (2023)
- [FedA3I: Annotation Quality-Aware Aggregation for Federated Medical Image Segmentation against Heterogeneous Annotation Noise](https://mlsanity.com/paper/2312.12838/) (2023)
- [Real World Federated Learning with a Knowledge Distilled Transformer for Cardiac CT Imaging](https://mlsanity.com/paper/2407.07557/) (2024)
- [FUNAvg: Federated Uncertainty Weighted Averaging for Datasets with Diverse Labels](https://mlsanity.com/paper/2407.07488/) (2024)
- [Federated nnU-Net for Privacy-Preserving Medical Image Segmentation](https://mlsanity.com/paper/2503.02549/) (2025)
- [Federated Self-Supervised Learning for One-Shot Cross-Modal and Cross-Imaging Technique Segmentation](https://mlsanity.com/paper/2503.23507/) (2025)
- [Transfer learning from a sparsely annotated dataset of 3D medical images](https://mlsanity.com/paper/2311.05032/) (2023)
- [Anatomical 3D Style Transfer Enabling Efficient Federated Learning with Extremely Low Communication Costs](https://mlsanity.com/paper/2410.20102/) (2024)
- [Federated Learning with Partially Labeled Data: A Conditional Distillation Approach](https://mlsanity.com/paper/2412.18833/) (2024)
- [Asynchronous Federated Continual Segmentation with Evolving Clients and Label Spaces](https://mlsanity.com/paper/2503.15414/) (2025)
- [Dynamic Allocation Hypernetwork with Adaptive Model Recalibration for FCL](https://mlsanity.com/paper/2503.18064/) (2025)
- [FIVA: Federated Inverse Variance Averaging for Universal CT Segmentation with Uncertainty Estimation](https://mlsanity.com/paper/2508.09196/) (2025)
- [Self-Supervised Federated Learning under Data Heterogeneity for Label-Scarce Diatom Classification](https://mlsanity.com/paper/2603.29633/) (2026)
- [Overcoming data scarcity through multi-center federated learning for organs-at-risk segmentation in pediatric upper abdominal radiotherapy](https://mlsanity.com/paper/2605.06820/) (2026)
- [FedMust: Semi-supervised Multi-task Student-Teacher Federated Learning for Multi-organ CT Segmentation](https://mlsanity.com/paper/2609.24627/) (2026)

## Abstract (arXiv)

Federated learning is an emerging paradigm allowing large-scale decentralized learning without sharing data across different data owners, which helps address the concern of data privacy in medical image analysis. However, the requirement for label consistency across clients by the existing methods largely narrows its application scope. In practice, each clinical site may only annotate certain organs of interest with partial or no overlap with other sites. Incorporating such partially labeled data into a unified federation is an unexplored problem with clinical significance and urgency. This work tackles the challenge by using a novel federated multi-encoding U-Net (Fed-MENU) method for multi-organ segmentation. In our method, a multi-encoding U-Net (MENU-Net) is proposed to extract organ-specific features through different encoding sub-networks. Each sub-network can be seen as an expert of a specific organ and trained for that client. Moreover, to encourage the organ-specific features extracted by different sub-networks to be informative and distinctive, we regularize the training of the MENU-Net by designing an auxiliary generic decoder (AGD). Extensive experiments on six public abdominal CT datasets show that our Fed-MENU method can effectively obtain a federated learning model using the partially labeled datasets with superior performance to other models trained by either localized or centralized learning methods. Source code is publicly available at https://github.com/DIAL-RPI/Fed-MENU.