# Where to Begin? From Random to Foundation Model Instructed Initialization in Federated Learning for Medical Image Segmentation

arXiv: https://arxiv.org/abs/2311.15463 · HTML: https://mlsanity.com/paper/2311.15463/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 15 references that are in the map.
- First posted on arXiv 27 Nov 2023.

## Builds on (most cited first, 11 of 11)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [SAM-Med2D](https://mlsanity.com/paper/2308.16184/) (2023)
- [On the Importance and Applicability of Pre-Training for Federated Learning](https://mlsanity.com/paper/2206.11488/) (2022)
- [Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learning](https://mlsanity.com/paper/2210.08090/) (2022)
- [Knowledge distillation with Segment Anything (SAM) model for Planetary Geological Mapping](https://mlsanity.com/paper/2305.07586/) (2023)
- [Segment Anything Model is a Good Teacher for Local Feature Learning](https://mlsanity.com/paper/2309.16992/) (2023)
- [Recurrent Aggregation Learning for Multi-View Echocardiographic Sequences Segmentation](https://mlsanity.com/paper/1907.11292/) (2019)
- [Explainable COVID-19 Infections Identification and Delineation Using Calibrated Pseudo Labels](https://mlsanity.com/paper/2202.07422/) (2022)
- [Data-Free Distillation Improves Efficiency and Privacy in Federated Thorax Disease Analysis](https://mlsanity.com/paper/2310.18346/) (2023)

## Built on by (most cited first, 3 of 3)

- [From Challenges and Pitfalls to Recommendations and Opportunities: Implementing Federated Learning in Healthcare](https://mlsanity.com/paper/2409.09727/) (2024)
- [Federated Client-tailored Adapter for Medical Image Segmentation](https://mlsanity.com/paper/2504.18020/) (2025)
- [Heterogeneity-Aware Coordination for Federated Learning via Stitching Pre-trained blocks](https://mlsanity.com/paper/2409.07202/) (2024)

## Abstract (arXiv)

In medical image analysis, Federated Learning (FL) stands out as a key technology that enables privacy-preserved, decentralized data processing, crucial for handling sensitive medical data. Currently, most FL models employ random initialization, which has been proven effective in various instances. However, given the unique challenges posed by non-IID (independently and identically distributed) data in FL, we propose a novel perspective: exploring the impact of using the foundation model with enormous pre-trained knowledge, such as the Segment Anything Model (SAM), as an instructive teacher for FL model initialization in medical image segmentation task. This work for the first time attempts to utilize the foundation model as an instructive teacher for initialization in FL, assessing its impact on the performance of FL models, especially in non-IID data scenarios. Our empirical evaluation on chest x-ray lung segmentation showcases that FL with foundation model instructed initialization not only achieves faster convergence but also improves performance in complex data contexts. These findings offer a new perspective for model initialization in FL.