mlsanityData to 14 Oct 2026

Masked Image Modeling Advances 3D Medical Image Analysis

arXiv 2204.11716 ↗First posted 25 Apr 2022cs.CV cs.AI cs.LGcited by 72

Not on the map. Why?

Abstract (arXiv)

Recently, masked image modeling (MIM) has gained considerable attention due to its capacity to learn from vast amounts of unlabeled data and has been demonstrated to be effective on a wide variety of vision tasks involving natural images. Meanwhile, the potential of self-supervised learning in modeling 3D medical images is anticipated to be immense due to the high quantities of unlabeled images, and the expense and difficulty of quality labels. However, MIM's applicability to medical images remains uncertain. In this paper, we demonstrate that masked image modeling approaches can also advance 3D medical images analysis in addition to natural images. We study how masked image modeling strategies leverage performance from the viewpoints of 3D medical image segmentation as a representative downstream task: i) when compared to naive contrastive learning, masked image modeling approaches accelerate the convergence of supervised training even faster (1.40×) and ultimately produce a higher dice score; ii) predicting raw voxel values with a high masking ratio and a relatively smaller patch size is non-trivial self-supervised pretext-task for medical images modeling; iii) a lightweight decoder or projection head design for reconstruction is powerful for masked image modeling on 3D medical images which speeds up training and reduce cost; iv) finally, we also investigate the effectiveness of MIM methods under different practical scenarios where different image resolutions and labeled data ratios are applied.

Key facts. Cited by 72 papers in the map (data through 14 Oct 2026). Builds on 42 of its 63 references that are in the map. First posted on arXiv 25 Apr 2022.

Builds on

42 of 63 references.

References · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  8. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  9. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  11. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  12. 2017Neural Discrete Representation Learningcited by 4,712
30 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  2. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  3. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  4. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  5. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  6. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  7. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  8. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  9. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  10. 2016Colorful Image Colorizationcited by 1,114
  11. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  12. 2021UNETR: Transformers for 3D Medical Image Segmentationcited by 959
  13. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 830
  14. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  15. 2020A Survey on Contrastive Self-supervised Learningcited by 554
  16. 2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 422
  17. 2019A large annotated medical image dataset for the development and evaluation of segmentation algorithmscited by 373
  18. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 368
  19. 2021Big Self-Supervised Models Advance Medical Image Classificationcited by 320
  20. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  21. 2019Med3D: Transfer Learning for 3D Medical Image Analysiscited by 288
  22. 2022Visual Attention Networkcited by 210
  23. 2016Learning Features by Watching Objects Movecited by 138
  24. 20203D Self-Supervised Methods for Medical Imagingcited by 134
  25. 2019A deep learning system for differential diagnosis of skin diseasescited by 118
  26. 2019Self-supervised Feature Learning for 3D Medical Images by Playing a Rubik's Cubecited by 76
  27. 2019Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Predictioncited by 70
  28. 2020Comparing to Learn: Surpassing ImageNet Pretraining on Radiographs By Comparing Image Representationscited by 69
  29. 2018Improving Cytoarchitectonic Segmentation of Human Brain Areas with Self-supervised Siamese Networkscited by 30
  30. 2021Imbalance-Aware Self-Supervised Learning for 3D Radiomic Representationscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 7; 2023: 9; 2024: 24; 2025: 13; 2026: 19.

72 papers.

Sort:

By area:

  1. 2024Exploring scalable medical image encoders beyond text supervisioncited by 132
  2. 2022A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyondcited by 58
  3. 2023Language Models are Few-shot Learners for Prognostic Predictioncited by 37
  4. 2024Large-Scale 3D Medical Image Pre-training with Geometric Context Priorscited by 36
  5. 2024Revisiting MAE pre-training for 3D medical image segmentationcited by 35
  6. 2025A generalizable 3D framework and model for self-supervised learning in medical imagingcited by 31
  7. 2024Masked Image Modeling: A Surveycited by 29
  8. 2023MIS-FM: 3D Medical Image Segmentation using Foundation Models Pretrained on a Large-Scale Unannotated Datasetcited by 26
  9. 2024An OpenMind for 3D medical vision self-supervised learningcited by 26
  10. 2024Multi-modal Vision Pre-training for Medical Image Analysiscited by 23
  11. 2023Advancing Volumetric Medical Image Segmentation via Global-Local Masked Autoencodercited by 21
  12. 2024MiM: Mask in Mask Self-Supervised Pre-Training for 3D Medical Image Analysiscited by 19
38 more
  1. 2023Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-trainingcited by 15
  2. 2023T3D: Advancing 3D Medical Vision-Language Pre-training by Learning Multi-View Visual Consistencycited by 14
  3. 2025Boosting Vision Semantic Density with Anatomy Normality Modeling for Medical Vision-language Pre-trainingcited by 14
  4. 2022GMML is All you Needcited by 12
  5. 20253D Foundation Model for Generalizable Disease Detection in Head Computed Tomographycited by 12
  6. 2024AnatoMask: Enhancing Medical Image Segmentation with Reconstruction-guided Self-maskingcited by 10
  7. 2024Improving Representation of High-frequency Components for Medical Visual Foundation Modelscited by 9
  8. 2023Deblurring Masked Autoencoder is Better Recipe for Ultrasound Image Recognitioncited by 8
  9. 2024MambaMIM: Pre-training Mamba with State Space Token Interpolation and its Application to Medical Image Segmentationcited by 7
  10. 2024Time-to-Event Pretraining for 3D Medical Imagingcited by 7
  11. 2022SB-SSL: Slice-Based Self-Supervised Transformers for Knee Abnormality Classification from MRIcited by 6
  12. 2022SPCXR: Self-supervised Pretraining using Chest X-rays Towards a Domain Specific Foundation Modelcited by 6
  13. 2024Primitive Geometry Segment Pre-training for 3D Medical Image Segmentationcited by 6
  14. 2024Self Pre-training with Topology- and Spatiality-aware Masked Autoencoders for 3D Medical Image Segmentationcited by 6
  15. 2025Distill CLIP (DCLIP): Enhancing Image-Text Retrieval via Cross-Modal Transformer Distillationcited by 6
  16. 2024Advancing High Resolution Vision-Language Models in Biomedicinecited by 5
  17. 2025Structure-aware Semantic Discrepancy and Consistency for 3D Medical Image Self-supervised Learningcited by 5
  18. 2023SRSNetwork: Siamese Reconstruction-Segmentation Networks based on Dynamic-Parameter Convolutioncited by 4
  19. 2023Building Universal Foundation Models for Medical Image Analysis with Spatially Adaptive Networkscited by 4
  20. 2024Real Time Multi Organ Classification on Computed Tomography Imagescited by 3
  21. 2025Adaptation of Foundation Models for Medical Image Analysis: Strategies, Challenges, and Future Directionscited by 3
  22. 2026Towards Brain MRI Foundation Models for the Clinic: Findings from the FOMO25 Challengecited by 3
  23. 2023A One Stop 3D Target Reconstruction and multilevel Segmentation Methodcited by 2
  24. 2024Self-supervised Learning of Dense Hierarchical Representations for Medical Image Segmentationcited by 2
  25. 2024Domain Aware Multi-Task Pretraining of 3D Swin Transformer for T1-weighted Brain MRIcited by 2
  26. 2025The Missing Piece: A Case for Pre-Training in 3D Medical Object Detectioncited by 2
  27. 2025Foundation Models in Biomedical Imaging: Turning Hype into Realitycited by 2
  28. 2022A Deep Learning Approach Using Masked Image Modeling for Reconstruction of Undersampled K-spacescited by 1
  29. 2022RGMIM: Region-Guided Masked Image Modeling for Learning Meaningful Representations from X-Ray Imagescited by 1
  30. 2022A Knowledge-based Learning Framework for Self-supervised Pre-training Towards Enhanced Recognition of Biomedical Microscopy Imagescited by 1
  31. 2024Sparse Anatomical Prompt Semi-Supervised Learning with Masked Image Modeling for CBCT Tooth Segmentationcited by 1
  32. 2024Self-supervised learning improves robustness of deep learning lung tumor segmentation to CT imaging differencescited by 1
  33. 2025MedCL: Learning Consistent Anatomy Distribution for Scribble-supervised Medical Image Segmentationcited by 1
  34. 2025Spatial-Aware Self-Supervision for Medical 3D Imaging with Multi-Granularity Observable Taskscited by 1
  35. 2025HU-based Foreground Masking for 3D Medical Masked Image Modelingcited by 1
  36. 2025Enhancing 3D Medical Image Understanding with Pretraining Aided by 2D Multimodal Large Language Modelscited by 1
  37. 2025Stochastic Siamese MAE Pretraining for Longitudinal Medical Imagescited by 1
  38. 2026MeDUET: Disentangled Unified Pretraining for 3D Medical Image Synthesis and Analysiscited by 1

50 most cited of 72 All 72 (JSON)