# USF-MAE: Ultrasound Self-Supervised Foundation Model with Masked Autoencoding

arXiv: https://arxiv.org/abs/2510.22990 · HTML: https://mlsanity.com/paper/2510.22990/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 74 references that are in the map.
- First posted on arXiv 27 Oct 2025.

## Builds on (most cited first, 19 of 19)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Deep Learning for Segmentation using an Open Large-Scale Dataset in 2D Echocardiography](https://mlsanity.com/paper/1908.06948/) (2019)
- [USCL: Pretraining Deep Ultrasound Image Diagnosis Model through Video Contrastive Representation Learning](https://mlsanity.com/paper/2011.13066/) (2020)
- [Tracked 3D Ultrasound and Deep Neural Network-based Thyroid Segmentation reduce Interobserver Variability in Thyroid Volumetry](https://mlsanity.com/paper/2108.10118/) (2021)
- [UltraSam: A Foundation Model for Ultrasound using Large Open-Access Segmentation Datasets](https://mlsanity.com/paper/2411.16222/) (2024)
- [GraphEcho: Graph-Driven Unsupervised Domain Adaptation for Echocardiogram Video Segmentation](https://mlsanity.com/paper/2309.11145/) (2023)
- [The Open Kidney Ultrasound Data Set](https://mlsanity.com/paper/2206.06657/) (2022)
- [ImageCHD: A 3D Computed Tomography Image Dataset for Classification of Congenital Heart Disease](https://mlsanity.com/paper/2101.10799/) (2021)
- [MMOTU: A Multi-Modality Ovarian Tumor Ultrasound Image Dataset for Unsupervised Cross-Domain Semantic Segmentation](https://mlsanity.com/paper/2207.06799/) (2022)
- [COVID-BLUeS -- A Prospective Study on the Value of AI in Lung Ultrasound Analysis](https://mlsanity.com/paper/2509.10556/) (2025)
- [A Comparative Survey of Vision Transformers for Feature Extraction in Texture Analysis](https://mlsanity.com/paper/2406.06136/) (2024)
- [CACTUS: An Open Dataset and Framework for Automated Cardiac Assessment and Classification of Ultrasound Images Using Deep Transfer Learning](https://mlsanity.com/paper/2503.05604/) (2025)
- [Algorithm guided outlining of 105 pancreatic cancer liver metastases in Ultrasound](https://mlsanity.com/paper/1710.02984/) (2017)
- [Nerve Block Target Localization and Needle Guidance for Autonomous Robotic Ultrasound Guided Regional Anesthesia](https://mlsanity.com/paper/2308.03717/) (2023)
- [A novel open-source ultrasound dataset with deep learning benchmarks for spinal cord injury localization and anatomical segmentation](https://mlsanity.com/paper/2409.16441/) (2024)
- [Segmentation of Muscularis Propria in Colon Histopathology Images Using Vision Transformers for Hirschsprung's Disease](https://mlsanity.com/paper/2412.20571/) (2024)

## Built on by (most cited first, 16 of 16)

- [Deep Learning Analysis of Prenatal Ultrasound for Identification of Ventriculomegaly](https://mlsanity.com/paper/2511.07827/) (2025)
- [Pretext Matters: An Empirical Study of SSL Methods in Medical Imaging](https://mlsanity.com/paper/2603.22649/) (2026)
- [Improved cystic hygroma detection from prenatal imaging using ultrasound-specific self-supervised representation learning](https://mlsanity.com/paper/2512.22730/) (2025)
- [US-JEPA: A Joint Embedding Predictive Architecture for Ultrasound](https://mlsanity.com/paper/2602.19322/) (2026)
- [Anatomy-Anchored Self-Supervision: Distilling Vision Foundation Models for Invariant Ultrasound Representation](https://mlsanity.com/paper/2605.25402/) (2026)
- [Automated Classification of First-Trimester Fetal Heart Views Using Ultrasound-Specific Self-Supervised Learning](https://mlsanity.com/paper/2512.24492/) (2025)
- [Beyond Independent Frames: Latent Attention Masked Autoencoders for Multi-View Echocardiography](https://mlsanity.com/paper/2604.15096/) (2026)
- [Automated Histopathologic Assessment of Hirschsprung Disease Using a Multi-Stage Vision Transformer Framework](https://mlsanity.com/paper/2511.20734/) (2025)
- [Benchmarking Self-Supervised Models for Cardiac Ultrasound View Classification](https://mlsanity.com/paper/2602.15339/) (2026)
- [PolarMAE: Efficient Fetal Ultrasound Pre-training via Semantic Screening and Polar-Guided Masking](https://mlsanity.com/paper/2604.15893/) (2026)
- [FADA: Accessible fetal ultrasound interpretation and annotation with a selectively distilled unified vision-language model](https://mlsanity.com/paper/2606.11106/) (2026)
- [FetSelect: Task-Specific Architectures and Self-Supervised Learning for Automated Fetal Ultrasound Frame Selection](https://mlsanity.com/paper/2606.22487/) (2026)
- [APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms](https://mlsanity.com/paper/2606.30577/) (2026)
- [Reading the Whole Heart: Latent-Attention Masked Autoencoders for Multimodal Cardiac Representation Learning](https://mlsanity.com/paper/2609.12035/) (2026)
- [Which Pretext Task Transfers? Self-Supervised Pretraining Objectives for Lung Ultrasound](https://mlsanity.com/paper/2609.16551/) (2026)
- [UltraBench 2: Towards Robust Evaluation of Vision Foundation Models on Ultrasound](https://mlsanity.com/paper/2609.28610/) (2026)

## Abstract (arXiv)

Ultrasound imaging is one of the most widely used diagnostic modalities, offering real-time, radiation-free assessment across diverse clinical domains. However, interpretation of ultrasound images remains challenging due to high noise levels, operator dependence, and limited field of view, resulting in substantial inter-observer variability. Current Deep Learning approaches are hindered by the scarcity of large labeled datasets and the domain gap between general and sonographic images, which limits the transferability of models pretrained on non-medical data. To address these challenges, we introduce the Ultrasound Self-Supervised Foundation Model with Masked Autoencoding (USF-MAE), the first large-scale self-supervised MAE framework pretrained exclusively on ultrasound data. The model was pre-trained on 370,000 2D and 3D ultrasound images curated from 46 open-source datasets, collectively termed OpenUS-46, spanning over twenty anatomical regions. This curated dataset has been made publicly available to facilitate further research and reproducibility. Using a Vision Transformer encoder-decoder architecture, USF-MAE reconstructs masked image patches, enabling it to learn rich, modality-specific representations directly from unlabeled data. The pretrained encoder was fine-tuned on three public downstream classification benchmarks: BUS-BRA (breast cancer), MMOTU-2D (ovarian tumors), and GIST514-DB (gastrointestinal stromal tumors). Across all tasks, USF-MAE consistently outperformed conventional CNN and ViT baselines, achieving F1-scores of 81.6%, 79.6%, and 82.4%, respectively. Despite not using labels during pretraining, USF-MAE approached the performance of the supervised foundation model UltraSam on breast cancer classification and surpassed it on the other tasks, demonstrating strong cross-anatomical generalization.