# MedMAE: A Self-Supervised Backbone for Medical Imaging Tasks

arXiv: https://arxiv.org/abs/2407.14784 · HTML: https://mlsanity.com/paper/2407.14784/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 55 references that are in the map.
- First posted on arXiv 20 Jul 2024.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [Contrastive learning of global and local features for medical image segmentation with limited annotations](https://mlsanity.com/paper/2006.10511/) (2020)
- [3D Self-Supervised Methods for Medical Imaging](https://mlsanity.com/paper/2006.03829/) (2020)
- [PGL: Prior-Guided Local Self-supervised Learning for 3D Medical Image Segmentation](https://mlsanity.com/paper/2011.12640/) (2020)
- [COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction](https://mlsanity.com/paper/2101.04909/) (2021)
- [Learning to diagnose from scratch by exploiting dependencies among labels](https://mlsanity.com/paper/1710.10501/) (2017)
- [Evolutionary Neural AutoML for Deep Learning](https://mlsanity.com/paper/1902.06827/) (2019)
- [MUXConv: Information Multiplexing in Convolutional Neural Networks](https://mlsanity.com/paper/2003.13880/) (2020)
- [Liver Fibrosis and NAS scoring from CT images using self-supervised learning and texture encoding](https://mlsanity.com/paper/2103.03761/) (2021)

## Built on by (most cited first, 5 of 5)

- [Visual Instruction Pretraining for Domain-Specific Foundation Models](https://mlsanity.com/paper/2509.17562/) (2025)
- [nnMobileNet++: Towards Efficient Hybrid Networks for Retinal Image Analysis](https://mlsanity.com/paper/2512.01273/) (2025)
- [Mask What Matters: Controllable Text-Guided Masking for Self-Supervised Medical Image Analysis](https://mlsanity.com/paper/2509.23054/) (2025)
- [MedDChest: A Content-Aware Multimodal Foundational Vision Model for Thoracic Imaging](https://mlsanity.com/paper/2511.04016/) (2025)
- [Balanced Soft mixture-of-expert model for Glaucoma Detection](https://mlsanity.com/paper/2607.25324/) (2026)

## Abstract (arXiv)

Medical imaging tasks are very challenging due to the lack of publicly available labeled datasets. Hence, it is difficult to achieve high performance with existing deep-learning models as they require a massive labeled dataset to be trained effectively. An alternative solution is to use pre-trained models and fine-tune them using the medical imaging dataset. However, all existing models are pre-trained using natural images, which is a completely different domain from that of medical imaging, which leads to poor performance due to domain shift. To overcome these problems, we propose a large-scale unlabeled dataset of medical images and a backbone pre-trained using the proposed dataset with a self-supervised learning technique called Masked autoencoder. This backbone can be used as a pre-trained model for any medical imaging task, as it is trained to learn a visual representation of different types of medical images. To evaluate the performance of the proposed backbone, we used four different medical imaging tasks. The results are compared with existing pre-trained models. These experiments show the superiority of our proposed backbone in medical imaging tasks.