mlsanityData to 7 Oct 2026

UniMiSS: Universal Medical Self-Supervised Learning via Breaking Dimensionality Barrier

arXiv 2112.09356 ↗First posted 17 Dec 2021cs.CVcited by 55

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) opens up huge opportunities for medical image analysis that is well known for its lack of annotations. However, aggregating massive (unlabeled) 3D medical images like computerized tomography (CT) remains challenging due to its high imaging cost and privacy restrictions. In this paper, we advocate bringing a wealth of 2D images like chest X-rays as compensation for the lack of 3D data, aiming to build a universal medical self-supervised representation learning framework, called UniMiSS. The following problem is how to break the dimensionality barrier, ie, making it possible to perform SSL with both 2D and 3D images? To achieve this, we design a pyramid U-like medical Transformer (MiT). It is composed of the switchable patch embedding (SPE) module and Transformers. The SPE module adaptively switches to either 2D or 3D patch embedding, depending on the input dimension. The embedded patches are converted into a sequence regardless of their original dimensions. The Transformers model the long-term dependencies in a sequence-to-sequence manner, thus enabling UniMiSS to learn representations from both 2D and 3D images. With the MiT as the backbone, we perform the UniMiSS in a self-distillation manner. We conduct expensive experiments on six 3D/2D medical image analysis tasks, including segmentation and classification. The results show that the proposed UniMiSS achieves promising performance on various downstream tasks, outperforming the ImageNet pre-training and other advanced SSL counterparts substantially. Code is available at defUrlFontrmsmallttfamily urlhttps://github.com/YtongXie/UniMiSS-code.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 39 of its 57 references that are in the map. First posted on arXiv 17 Dec 2021.

Builds on

39 of 57 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  10. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  11. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  12. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
27 more
  1. 2017Decoupled Weight Decay Regularizationcited by 1,722
  2. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  3. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  4. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  5. 2019Contrastive Multiview Codingcited by 1,297
  6. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  7. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  8. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  9. 2021UNETR: Transformers for 3D Medical Image Segmentationcited by 954
  10. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  11. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  12. 2020Contrastive learning of global and local features for medical image segmentation with limited annotationscited by 309
  13. 2021CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentationcited by 227
  14. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  15. 2016Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challengecited by 198
  16. 2017Colorization as a Proxy Task for Visual Understandingcited by 175
  17. 20203D Self-Supervised Methods for Medical Imagingcited by 134
  18. 2020MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Datacited by 121
  19. 2018Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Networkcited by 97
  20. 2020DoDNet: Learning to segment multi-organ and tumors from multiple partially labeled datasetscited by 94
  21. 2019Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentationcited by 88
  22. 2020Unpaired Multi-modal Segmentation via Knowledge Distillationcited by 81
  23. 2021Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contextscited by 59
  24. 2018A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocolscited by 49
  25. 2020PGL: Prior-Guided Local Self-supervised Learning for 3D Medical Image Segmentationcited by 36
  26. 2019A Mutual Bootstrapping Model for Automated Skin Lesion Segmentation and Classificationcited by 32
  27. 2021Dual-Teacher++: Exploiting Intra-domain and Inter-domain Knowledge with Reliable Transfer for Cardiac Segmentationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 18; 2024: 18; 2025: 12; 2026: 7.

55 papers.

Sort:

By area:

  1. 2023CLIP-Driven Universal Model for Organ Segmentation and Tumor Detectioncited by 224
  2. 2024AbdomenAtlas: A Large-Scale, Detailed-Annotated, & Multi-Center Dataset for Efficient Transfer Learning and Open Algorithmic Benchmarkingcited by 64
  3. 2023UniSeg: A Prompt-driven Universal Segmentation Model as well as A Strong Representation Learnercited by 46
  4. 2024nnMamba: 3D Biomedical Image Segmentation, Classification and Landmark Detection with State Space Modelcited by 45
  5. 2023Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learningcited by 34
  6. 2024Large-Scale 3D Medical Image Pre-training with Geometric Context Priorscited by 34
  7. 2023Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentationcited by 32
  8. 2024How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Modelcited by 32
  9. 2024Universal and Extensible Language-Vision Models for Organ Segmentation and Tumor Detection from Abdominal Computed Tomographycited by 31
  10. 2024CT-GLIP: 3D Grounded Language-Image Pretraining with CT Scans and Radiology Reports for Full-Body Scenarioscited by 29
  11. 2024Learning General-Purpose Biomedical Volume Representations using Randomized Synthesiscited by 27
  12. 2023Anatomical Invariance Modeling and Semantic Alignment for Self-supervised Learning in 3D Medical Image Analysiscited by 25
38 more
  1. 2025MedDINOv3: How to adapt vision foundation models for medical image segmentation?cited by 25
  2. 2023Advancing Volumetric Medical Image Segmentation via Global-Local Masked Autoencodercited by 19
  3. 2025How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks?cited by 19
  4. 2023Towards Foundation Models and Few-Shot Parameter-Efficient Fine-Tuning for Volumetric Organ Segmentationcited by 17
  5. 2024MiM: Mask in Mask Self-Supervised Pre-Training for 3D Medical Image Analysiscited by 17
  6. 2024Medical Multimodal Foundation Models in Clinical Diagnosis and Treatment: Applications, Challenges, and Future Directionscited by 17
  7. 2023Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-trainingcited by 15
  8. 2023ZePT: Zero-Shot Pan-Tumor Segmentation via Query-Disentangling and Self-Promptingcited by 15
  9. 2023Label-Efficient Deep Learning in Medical Image Analysis: Challenges and Future Directionscited by 14
  10. 2023T3D: Advancing 3D Medical Vision-Language Pre-training by Learning Multi-View Visual Consistencycited by 14
  11. 2023Segment Together: A Versatile Paradigm for Semi-Supervised Medical Image Segmentationcited by 12
  12. 2024ICHPro: Intracerebral Hemorrhage Prognosis Classification Via Joint-attention Fusion-based 3d Cross-modal Networkcited by 10
  13. 2024AnatoMask: Enhancing Medical Image Segmentation with Reconstruction-guided Self-maskingcited by 10
  14. 2023Attention Mechanisms in Medical Image Segmentation: A Surveycited by 9
  15. 2023MDViT: Multi-domain Vision Transformer for Small Medical Image Segmentation Datasetscited by 9
  16. 2023Gene-induced Multimodal Pre-training for Image-omic Classificationcited by 9
  17. 2024MambaMIM: Pre-training Mamba with State Space Token Interpolation and its Application to Medical Image Segmentationcited by 7
  18. 2024MedUniSeg: 2D and 3D Medical Image Segmentation via a Prompt-driven Universal Modelcited by 7
  19. 2025Towards Universal Text-driven CT Image Segmentationcited by 7
  20. 2025Foundation Models in Medical Imaging: A Review and Outlookcited by 7
  21. 2024Primitive Geometry Segment Pre-training for 3D Medical Image Segmentationcited by 6
  22. 2023Unified Medical Image Pre-training in Language-Guided Common Semantic Spacecited by 5
  23. 2025Continual Self-supervised Learning Considering Medical Domain Knowledge in Chest CT Imagescited by 5
  24. 2023Building Universal Foundation Models for Medical Image Analysis with Spatially Adaptive Networkscited by 4
  25. 2025MedVista3D: Vision-Language Modeling for Reducing Diagnostic Errors in 3D CT Disease Detection, Understanding and Reportingcited by 4
  26. 2024MLVICX: Multi-Level Variance-Covariance Exploration for Chest X-ray Self-Supervised Representation Learningcited by 3
  27. 2025Generalist Models in Medical Image Segmentation: A Survey and Performance Comparison with Task-Specific Approachescited by 2
  28. 2025M3Ret: Unleashing Zero-shot Multimodal Medical Image Retrieval via Self-Supervisioncited by 2
  29. 2023Parse and Recall: Towards Accurate Lung Nodule Malignancy Prediction like Radiologistscited by 1
  30. 2024Cross-Dimensional Medical Self-Supervised Representation Learning Based on a Pseudo-3D Transformationcited by 1
  31. 2024Low-Rank Continual Pyramid Vision Transformer: Incrementally Segment Whole-Body Organs in CT with Light-Weighted Adaptationcited by 1
  32. 2026Anatomy Contextualized Adaptation of CT Foundation Modelscited by 1
  33. 2024Abnormality-Driven Representation Learning for Radiology Imagingcited by 0
  34. 2025AdaViT: Adaptive Vision Transformer for Flexible Pretrain and Finetune with Variable 3D Medical Image Modalitiescited by 0
  35. 2025DiSSECT: Structuring Transfer-Ready Medical Image Representations through Discrete Self-Supervisioncited by 0
  36. 2025DuPLUS: Dual-Prompt Vision-Language Framework for Universal Medical Image Segmentation and Prognosiscited by 0
  37. 2025MoME: Mixture of Visual Language Medical Experts for Medical Imaging Segmentationcited by 0
  38. 2026FDIF: Formula-Driven supervised Learning with Implicit Functions for 3D Medical Image Segmentationcited by 0

50 most cited of 55 All 55 (JSON)