# AMAES: Augmented Masked Autoencoder Pretraining on Public Brain MRI Data for 3D-Native Segmentation

arXiv: https://arxiv.org/abs/2408.00640 · HTML: https://mlsanity.com/paper/2408.00640/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 16 of 16)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [Contrastive learning of global and local features for medical image segmentation with limited annotations](https://mlsanity.com/paper/2006.10511/) (2020)
- [nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation](https://mlsanity.com/paper/2404.09556/) (2024)
- [MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation](https://mlsanity.com/paper/2303.09975/) (2023)
- [Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge](https://mlsanity.com/paper/1904.00682/) (2019)
- [ISLES 2022: A multi-center magnetic resonance imaging stroke lesion segmentation dataset](https://mlsanity.com/paper/2206.06694/) (2022)
- [Yucca: A Deep Learning Framework For Medical Image Analysis](https://mlsanity.com/paper/2407.19888/) (2024)

## Built on by (most cited first, 17 of 17)

- [Revisiting MAE pre-training for 3D medical image segmentation](https://mlsanity.com/paper/2410.23132/) (2024)
- [An OpenMind for 3D medical vision self-supervised learning](https://mlsanity.com/paper/2412.17041/) (2024)
- [Comprehensive language-image pre-training for 3D medical image understanding](https://mlsanity.com/paper/2510.15042/) (2025)
- [A large-scale heterogeneous 3D magnetic resonance brain imaging dataset for self-supervised learning](https://mlsanity.com/paper/2506.14432/) (2025)
- [Global and Local Contrastive Learning for Joint Representations from Cardiac MRI and ECG](https://mlsanity.com/paper/2506.20683/) (2025)
- [BrainDINO: A Brain MRI Foundation Model for Generalizable Clinical Representation Learning](https://mlsanity.com/paper/2604.27277/) (2026)
- [Towards Brain MRI Foundation Models for the Clinic: Findings from the FOMO25 Challenge](https://mlsanity.com/paper/2604.11679/) (2026)
- [Self is the Best Learner: CT-free Ultra-Low-Dose PET Organ Segmentation via Collaborating Denoising and Segmentation Learning](https://mlsanity.com/paper/2503.03786/) (2025)
- [From 100,000+ images to winning the first brain MRI foundation model challenges: Sharing lessons and models](https://mlsanity.com/paper/2601.13166/) (2026)
- [Revisiting CLIP: Efficient Alignment of 3D MRI and Tabular Data using Domain-Specific Foundation Models](https://mlsanity.com/paper/2501.14051/) (2025)
- [A Foundation Model for Brain MRI with Dynamic Modality Integration](https://mlsanity.com/paper/2511.03014/) (2025)
- [brat: Aligned Multi-View Embeddings for Brain MRI Analysis](https://mlsanity.com/paper/2512.18679/) (2025)
- [Bayesian Integration of Nonlinear Incomplete Clinical Data](https://mlsanity.com/paper/2602.01924/) (2026)
- [Contrastive Joint-Embedding Prediction for Representation Learning in Structural MRI](https://mlsanity.com/paper/2607.11962/) (2026)
- [BrainNext: A General-Purpose Self-Supervised Foundation Model for Brain MRI Analysis](https://mlsanity.com/paper/2607.17782/) (2026)
- [Curia-MAE: Multi-Modal Multi-Anatomy MAE Pre-Training for 3D Medical Image Segmentation](https://mlsanity.com/paper/2608.05844/) (2026)
- [nnFoundation: 3D Foundation Models for Radiology](https://mlsanity.com/paper/2609.26924/) (2026)

## Abstract (arXiv)

This study investigates the impact of self-supervised pretraining of 3D semantic segmentation models on a large-scale, domain-specific dataset. We introduce BRAINS-45K, a dataset of 44,756 brain MRI volumes from public sources, the largest public dataset available, and revisit a number of design choices for pretraining modern segmentation architectures by simplifying and optimizing state-of-the-art methods, and combining them with a novel augmentation strategy. The resulting AMAES framework is based on masked-image-modeling and intensity-based augmentation reversal and balances memory usage, runtime, and finetuning performance. Using the popular U-Net and the recent MedNeXt architecture as backbones, we evaluate the effect of pretraining on three challenging downstream tasks, covering single-sequence, low-resource settings, and out-of-domain generalization. The results highlight that pretraining on the proposed dataset with AMAES significantly improves segmentation performance in the majority of evaluated cases, and that it is beneficial to pretrain the model with augmentations, despite pretraing on a large-scale dataset. Code and model checkpoints for reproducing results, as well as the BRAINS-45K dataset are available at urlhttps://github.com/asbjrnmunk/amaes.