# Self-supervised Feature Learning via Exploiting Multi-modal Data for Retinal Disease Diagnosis

arXiv: https://arxiv.org/abs/2007.11067 · HTML: https://mlsanity.com/paper/2007.11067/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2020.

## Built on by (most cited first, 30 of 47)

- [On the Challenges and Perspectives of Foundation Models for Medical Image Analysis](https://mlsanity.com/paper/2306.05705/) (2023)
- [Recent advances and clinical applications of deep learning in medical image analysis](https://mlsanity.com/paper/2105.13381/) (2021)
- [Self-supervised learning methods and applications in medical imaging analysis: A survey](https://mlsanity.com/paper/2109.08685/) (2021)
- [Self-supervised driven consistency training for annotation efficient histopathology image analysis](https://mlsanity.com/paper/2102.03897/) (2021)
- [Segment Anything Model for Medical Images?](https://mlsanity.com/paper/2304.14660/) (2023)
- [LE-UDA: Label-efficient unsupervised domain adaptation for medical image segmentation](https://mlsanity.com/paper/2212.02078/) (2022)
- [Lesion-based Contrastive Learning for Diabetic Retinopathy Grading from Fundus Images](https://mlsanity.com/paper/2107.08274/) (2021)
- [Uni4Eye: Unified 2D and 3D Self-supervised Pre-training via Masked Image Modeling Transformer for Ophthalmic Image Classification](https://mlsanity.com/paper/2203.04614/) (2022)
- [Label-Efficient Deep Learning in Medical Image Analysis: Challenges and Future Directions](https://mlsanity.com/paper/2303.12484/) (2023)
- [Anatomy-aware Self-supervised Learning for Anomaly Detection in Chest Radiographs](https://mlsanity.com/paper/2205.04282/) (2022)
- [A Review of Predictive and Contrastive Self-supervised Learning for Medical Images](https://mlsanity.com/paper/2302.05043/) (2023)
- [SSiT: Saliency-guided Self-supervised Image Transformer for Diabetic Retinopathy Grading](https://mlsanity.com/paper/2210.10969/) (2022)
- [Robust Box Prompt based SAM for Medical Image Segmentation](https://mlsanity.com/paper/2407.21284/) (2024)
- [Free Lunch for Surgical Video Understanding by Distilling Self-Supervisions](https://mlsanity.com/paper/2205.09292/) (2022)
- [GTP-4o: Modality-prompted Heterogeneous Graph Learning for Omni-modal Biomedical Representation](https://mlsanity.com/paper/2407.05540/) (2024)
- [MVCNet: Multiview Contrastive Network for Unsupervised Representation Learning for 3D CT Lesions](https://mlsanity.com/paper/2108.07662/) (2021)
- [Applications of Explainable AI for 6G: Technical Aspects, Use Cases, and Research Challenges](https://mlsanity.com/paper/2112.04698/) (2021)
- [Generating and Weighting Semantically Consistent Sample Pairs for Ultrasound Contrastive Learning](https://mlsanity.com/paper/2212.04097/) (2022)
- [Confidence-aware multi-modality learning for eye disease screening](https://mlsanity.com/paper/2405.18167/) (2024)
- [Semi-weakly Supervised Contrastive Representation Learning for Retinal Fundus Images](https://mlsanity.com/paper/2108.02122/) (2021)
- [A Survey of Multimodal Ophthalmic Diagnostics: From Task-Specific Approaches to Foundational Models](https://mlsanity.com/paper/2508.03734/) (2025)
- [Unsupervised Local Discrimination for Medical Images](https://mlsanity.com/paper/2108.09440/) (2021)
- [DRStageNet: Deep Learning for Diabetic Retinopathy Staging from Fundus Images](https://mlsanity.com/paper/2312.14891/) (2023)
- [Joint Self-Supervised and Supervised Contrastive Learning for Multimodal MRI Data: Towards Predicting Abnormal Neurodevelopment](https://mlsanity.com/paper/2312.15064/) (2023)
- [Weakly-supervised detection of AMD-related lesions in color fundus images using explainable deep learning](https://mlsanity.com/paper/2212.00565/) (2022)
- [A Novel Collaborative Self-Supervised Learning Method for Radiomic Data](https://mlsanity.com/paper/2302.09807/) (2023)
- [Learning Self-Supervised Representations for Label Efficient Cross-Domain Knowledge Transfer on Diabetic Retinopathy Fundus Images](https://mlsanity.com/paper/2304.11168/) (2023)
- [An Attentive-based Generative Model for Medical Image Synthesis](https://mlsanity.com/paper/2306.01562/) (2023)
- [Thyroid ultrasound diagnosis improvement via multi-view self-supervised learning and two-stage pre-training](https://mlsanity.com/paper/2402.11497/) (2024)
- [Enabling Real-Time Colonoscopic Polyp Segmentation on Commodity CPUs via Ultra-Lightweight Architecture](https://mlsanity.com/paper/2602.04381/) (2026)

## Abstract (arXiv)

The automatic diagnosis of various retinal diseases from fundus images is important to support clinical decision-making. However, developing such automatic solutions is challenging due to the requirement of a large amount of human-annotated data. Recently, unsupervised/self-supervised feature learning techniques receive a lot of attention, as they do not need massive annotations. Most of the current self-supervised methods are analyzed with single imaging modality and there is no method currently utilize multi-modal images for better results. Considering that the diagnostics of various vitreoretinal diseases can greatly benefit from another imaging modality, e.g., FFA, this paper presents a novel self-supervised feature learning method by effectively exploiting multi-modal data for retinal disease diagnosis. To achieve this, we first synthesize the corresponding FFA modality and then formulate a patient feature-based softmax embedding objective. Our objective learns both modality-invariant features and patient-similarity features. Through this mechanism, the neural network captures the semantically shared information across different modalities and the apparent visual similarity between patients. We evaluate our method on two public benchmark datasets for retinal disease diagnosis. The experimental results demonstrate that our method clearly outperforms other self-supervised feature learning methods and is comparable to the supervised baseline.