mlsanityData to 13 Oct 2026

Uni4Eye: Unified 2D and 3D Self-supervised Pre-training via Masked Image Modeling Transformer for Ophthalmic Image Classification

arXiv 2203.04614 ↗First posted 9 Mar 2022eess.IV cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

A large-scale labeled dataset is a key factor for the success of supervised deep learning in computer vision. However, a limited number of annotated data is very common, especially in ophthalmic image analysis, since manual annotation is time-consuming and labor-intensive. Self-supervised learning (SSL) methods bring huge opportunities for better utilizing unlabeled data, as they do not need massive annotations. With an attempt to use as many as possible unlabeled ophthalmic images, it is necessary to break the dimension barrier, simultaneously making use of both 2D and 3D images. In this paper, we propose a universal self-supervised Transformer framework, named Uni4Eye, to discover the inherent image property and capture domain-specific feature embedding in ophthalmic images. Uni4Eye can serve as a global feature extractor, which builds its basis on a Masked Image Modeling task with a Vision Transformer (ViT) architecture. We employ a Unified Patch Embedding module to replace the origin patch embedding module in ViT for jointly processing both 2D and 3D input images. Besides, we design a dual-branch multitask decoder module to simultaneously perform two reconstruction tasks on the input image and its gradient map, delivering discriminative representations for better convergence. We evaluate the performance of our pre-trained Uni4Eye encoder by fine-tuning it on six downstream ophthalmic image classification tasks. The superiority of Uni4Eye is successfully established through comparisons to other state-of-the-art SSL pre-training methods.

Key facts. Cited by 18 papers in the map (data through 13 Oct 2026). Builds on 24 of its 32 references that are in the map. First posted on arXiv 9 Mar 2022.

Builds on

24 of 32 references.

References · 24

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,043
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,009
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,492
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,364
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,426
  7. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,368
  8. 2017Decoupled Weight Decay Regularizationcited by 1,788
  9. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,590
  10. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 574
  11. 2020Contrastive learning of global and local features for medical image segmentation with limited annotationscited by 309
  12. 2019Med3D: Transfer Learning for 3D Medical Image Analysiscited by 284
12 more
  1. 2019Large Scale Adversarial Representation Learningcited by 239
  2. 2019Unsupervised Embedding Learning via Invariant and Spreading Instance Featurecited by 231
  3. 20203D Self-Supervised Methods for Medical Imagingcited by 134
  4. 2021SiT: Self-supervised vIsion Transformercited by 99
  5. 2019Self-supervised Feature Learning for 3D Medical Images by Playing a Rubik's Cubecited by 76
  6. 2021Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contextscited by 59
  7. 2020Self-supervised Feature Learning via Exploiting Multi-modal Data for Retinal Disease Diagnosiscited by 47
  8. 2021LongReMix: Robust Learning with High Confidence Samples in a Noisy Label Environmentcited by 40
  9. 2021Lesion-based Contrastive Learning for Diabetic Retinopathy Grading from Fundus Imagescited by 24
  10. 2021BSDA-Net: A Boundary Shape and Distance Aware Joint Learning Framework for Segmenting and Classifying OCTA Imagescited by 17
  11. 2022COROLLA: An Efficient Multi-Modality Fusion Framework with Supervised Contrastive Learning for Glaucoma Gradingcited by 10
  12. 2022JOINED : Prior Guided Multi-task Learning for Joint Optic Disc/Cup Segmentation and Fovea Detectioncited by 6

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 8; 2025: 1; 2026: 2.

18 papers.

Sort:

  1. 2023Continual Self-supervised Learning: Towards Universal Multi-modal Medical Data Representation Learningcited by 34
  2. 2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectivescited by 33
  3. 2024Masked Image Modeling: A Surveycited by 28
  4. 2023Label-Efficient Deep Learning in Medical Image Analysis: Challenges and Future Directionscited by 15
  5. 2022SSiT: Saliency-guided Self-supervised Image Transformer for Diabetic Retinopathy Gradingcited by 9
  6. 2024MedUniSeg: 2D and 3D Medical Image Segmentation via a Prompt-driven Universal Modelcited by 7
  7. 2023Revisiting Computer-Aided Tuberculosis Diagnosiscited by 6
  8. 2024Confidence-aware multi-modality learning for eye disease screeningcited by 5
  9. 2024Text-guided Foundation Model Adaptation for Long-Tailed Medical Image Classificationcited by 5
  10. 2024ADAPT: Alzheimer Diagnosis through Adaptive Profiling Transformerscited by 3
  11. 2024From Few to More: Scribble-based Medical Image Segmentation via Masked Context Modeling and Continuous Pseudo Labelscited by 3
  12. 2025A Survey of Multimodal Ophthalmic Diagnostics: From Task-Specific Approaches to Foundational Modelscited by 3
6 more
  1. 2023SF2Former: Amyotrophic Lateral Sclerosis Identification From Multi-center MRI Data Using Spatial and Frequency Fusion Transformercited by 1
  2. 2023Reliable Multimodality Eye Disease Screening via Mixture of Student's t Distributionscited by 1
  3. 2024PoCo: A Self-Supervised Approach via Polar Transformation Based Progressive Contrastive Learning for Ophthalmic Disease Diagnosiscited by 0
  4. 2024DS-ViT: Dual-Stream Vision Transformer for Cross-Task Distillation in Alzheimer's Early Diagnosiscited by 0
  5. 2026Context-Aware Asymmetric Ensembling for Interpretable Retinopathy of Prematurity Screening via Active Query and Vascular Attentioncited by 0
  6. 2026PolarMAE: Efficient Fetal Ultrasound Pre-training via Semantic Screening and Polar-Guided Maskingcited by 0