# From Pretraining to Privacy: Federated Ultrasound Foundation Model with Self-Supervised Learning

arXiv: https://arxiv.org/abs/2411.16380 · HTML: https://mlsanity.com/paper/2411.16380/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 77 references that are in the map.
- First posted on arXiv 25 Nov 2024.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Deep Learning for Segmentation using an Open Large-Scale Dataset in 2D Echocardiography](https://mlsanity.com/paper/1908.06948/) (2019)
- [Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-train](https://mlsanity.com/paper/2306.16741/) (2023)
- [SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound](https://mlsanity.com/paper/1612.05601/) (2016)
- [USFM: A Universal Ultrasound Foundation Model Generalized to Tasks and Organs towards Label Efficient Image Analysis](https://mlsanity.com/paper/2401.00153/) (2023)
- [AAU-net: An Adaptive Attention U-net for Breast Lesions Segmentation in Ultrasound Images](https://mlsanity.com/paper/2204.12077/) (2022)
- [A New Dataset and A Baseline Model for Breast Lesion Detection in Ultrasound Videos](https://mlsanity.com/paper/2207.00141/) (2022)
- [GraphEcho: Graph-Driven Unsupervised Domain Adaptation for Echocardiogram Video Segmentation](https://mlsanity.com/paper/2309.11145/) (2023)
- [RadFormer: Transformers with Global-Local Attention for Interpretable and Accurate Gallbladder Cancer Detection](https://mlsanity.com/paper/2211.04793/) (2022)
- [FPUS23: An Ultrasound Fetus Phantom Dataset with Deep Neural Network Evaluations for Fetus Orientations, Fetal Planes, and Anatomical Features](https://mlsanity.com/paper/2303.07852/) (2023)
- [FocusMAE: Gallbladder Cancer Detection from Ultrasound Videos with Focused Masked Autoencoders](https://mlsanity.com/paper/2403.08848/) (2024)
- [You Can Mask More For Extremely Low-Bitrate Image Compression](https://mlsanity.com/paper/2306.15561/) (2023)
- [Recognizing Focal Liver Lesions in Contrast-Enhanced Ultrasound with Discriminatively Trained Spatio-Temporal Model](https://mlsanity.com/paper/1502.00750/) (2015)
- [Towards a Benchmark for Colorectal Cancer Segmentation in Endorectal Ultrasound Videos: Dataset and Model Development](https://mlsanity.com/paper/2408.10067/) (2024)

## Built on by (most cited first, 15 of 15)

- [Vision Foundation Models in Medical Image Analysis: Advances and Challenges](https://mlsanity.com/paper/2502.14584/) (2025)
- [General Methods Make Great Domain-specific Foundation Models: A Case-study on Fetal Ultrasound](https://mlsanity.com/paper/2506.19552/) (2025)
- [FUGC: Benchmarking Semi-Supervised Learning Methods for Cervical Segmentation](https://mlsanity.com/paper/2601.15572/) (2026)
- [Vision Foundation Models in Radiology: A Scoping Review of Data, Methodology, Evaluation and Clinical Translation](https://mlsanity.com/paper/2607.07219/) (2026)
- [Multimodal, Multi-Disease Medical Imaging Foundation Model (MerMED-FM)](https://mlsanity.com/paper/2507.00185/) (2025)
- [Diagnostic Performance of Universal-Learning Ultrasound AI Across Multiple Organs and Tasks: the UUSIC25 Challenge](https://mlsanity.com/paper/2512.17279/) (2025)
- [AURORA: Adaptive Unified Representation for Robust Ultrasound Analysis](https://mlsanity.com/paper/2603.19364/) (2026)
- [MVis-Fold: A Three-Dimensional Microvascular Structure Inference Model for Super-Resolution Ultrasound](https://mlsanity.com/paper/2604.04477/) (2026)
- [FADA: Accessible fetal ultrasound interpretation and annotation with a selectively distilled unified vision-language model](https://mlsanity.com/paper/2606.11106/) (2026)
- [APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms](https://mlsanity.com/paper/2606.30577/) (2026)
- [FedCC: A Low-Resource Federated Adaptation of Foundation Models for Robust Corpus Callosum localization in Fetal Ultrasound Images](https://mlsanity.com/paper/2607.18283/) (2026)
- [IQ-JEPA: A Joint-Embedding Predictive Architecture with a Hermitian Vision Transformer for Sound Speed and Attenuation Estimation from Ultrasound IQ Data](https://mlsanity.com/paper/2607.22351/) (2026)
- [A generalizable structural brain MRI foundation model built through dual-priority federated pretraining](https://mlsanity.com/paper/2609.27611/) (2026)
- [UltraBench 2: Towards Robust Evaluation of Vision Foundation Models on Ultrasound](https://mlsanity.com/paper/2609.28610/) (2026)
- [A Concept-informed Interpretable Artificial Intelligence Framework for Breast Ultrasound Diagnosis](https://mlsanity.com/paper/2610.16128/) (2026)

## Abstract (arXiv)

Ultrasound imaging is widely used in clinical diagnosis due to its non-invasive nature and real-time capabilities. However, traditional ultrasound diagnostics relies heavily on physician expertise and is often hampered by suboptimal image quality, leading to potential diagnostic errors. While artificial intelligence (AI) offers a promising solution to enhance clinical diagnosis by detecting abnormalities across various imaging modalities, existing AI methods for ultrasound face two major challenges. First, they typically require vast amounts of labeled medical data, raising serious concerns regarding patient privacy. Second, most models are designed for specific tasks, which restricts their broader clinical utility. To overcome these challenges, we present UltraFedFM, an innovative privacy-preserving ultrasound foundation model. UltraFedFM is collaboratively pre-trained using federated learning across 16 distributed medical institutions in 9 countries, leveraging a dataset of over 1 million ultrasound images covering 19 organs and 10 ultrasound modalities. This extensive and diverse data, combined with a secure training framework, enables UltraFedFM to exhibit strong generalization and diagnostic capabilities. It achieves an average area under the receiver operating characteristic curve (AUROC) of 0.927 for disease diagnosis and a dice similarity coefficient (DSC) of 0.878 for lesion segmentation. Notably, UltraFedFM surpasses the diagnostic accuracy of mid-level ultrasonographers (4-8 years of experience) and matches the performance of expert-level sonographers (10+ years of experience) in the joint diagnosis of 8 common systemic diseases.c These findings indicate that UltraFedFM can significantly enhance clinical diagnostics while safeguarding patient privacy, marking a significant advancement in AI-driven ultrasound imaging for future clinical applications.