# Clinically Labeled Contrastive Learning for OCT Biomarker Classification

arXiv: https://arxiv.org/abs/2305.15154 · HTML: https://mlsanity.com/paper/2305.15154/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 55 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Contrastive Learning of Medical Visual Representations from Paired Images and Text](https://mlsanity.com/paper/2010.00747/) (2020)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [Big Self-Supervised Models Advance Medical Image Classification](https://mlsanity.com/paper/2101.05224/) (2021)
- [When Does Contrastive Visual Representation Learning Work?](https://mlsanity.com/paper/2105.05837/) (2021)
- [A Review of Challenges and Opportunities in Machine Learning for Health](https://mlsanity.com/paper/1806.00388/) (2018)
- [MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models](https://mlsanity.com/paper/2010.05352/) (2020)
- [MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretation](https://mlsanity.com/paper/2102.10663/) (2021)
- [Patient Contrastive Learning: a Performant, Expressive, and Practical Approach to ECG Modeling](https://mlsanity.com/paper/2104.04569/) (2021)
- [OLIVES Dataset: Ophthalmic Labels for Investigating Visual Eye Semantics](https://mlsanity.com/paper/2209.11195/) (2022)
- [Modeling Disease Progression In Retinal OCTs With Longitudinal Self-Supervised Learning](https://mlsanity.com/paper/1910.09420/) (2019)
- [Contrastive Cross-site Learning with Redesigned Net for COVID-19 CT Classification](https://mlsanity.com/paper/2009.07652/) (2020)
- [Deep Learning based Retinal OCT Segmentation](https://mlsanity.com/paper/1801.09749/) (2018)
- [Volumetric Supervised Contrastive Learning for Seismic Semantic Segmentation](https://mlsanity.com/paper/2206.08158/) (2022)
- [Contrastive Reasoning in Neural Networks](https://mlsanity.com/paper/2103.12329/) (2021)
- [Structure Label Prediction Using Similarity-Based Retrieval and Weakly-Supervised Label Mapping](https://mlsanity.com/paper/1905.06528/) (2019)
- [Relative Afferent Pupillary Defect Screening through Transfer Learning](https://mlsanity.com/paper/1908.02300/) (2019)
- [Contrastive Cross-Modal Pre-Training: A General Strategy for Small Sample Medical Imaging](https://mlsanity.com/paper/2010.03060/) (2020)
- [Multi-Modal Learning Using Physicians Diagnostics for Optical Coherence Tomography Classification](https://mlsanity.com/paper/2203.10622/) (2022)

## Built on by (most cited first, 9 of 9)

- [FOCAL: A Cost-Aware Video Dataset for Active Learning](https://mlsanity.com/paper/2311.10591/) (2023)
- [HEX: Hierarchical Emergence Exploitation in Self-Supervised Algorithms](https://mlsanity.com/paper/2410.23200/) (2024)
- [Hierarchical and Multimodal Data for Daily Activity Understanding](https://mlsanity.com/paper/2504.17696/) (2025)
- [A Large-scale Benchmark on Geological Fault Delineation Models: Domain Shift, Training Dynamics, Generalizability, Evaluation and Inferential Behavior](https://mlsanity.com/paper/2505.08585/) (2025)
- [AdaDim: Dimensionality Adaptation for SSL Representational Dynamics](https://mlsanity.com/paper/2505.12576/) (2025)
- [Clinical Trial Active Learning](https://mlsanity.com/paper/2307.11209/) (2023)
- [VOICE: Variance of Induced Contrastive Explanations to quantify Uncertainty in Neural Network Interpretability](https://mlsanity.com/paper/2406.00573/) (2024)
- [Leveraging Complementary Attention maps in vision transformers for OCT image analysis](https://mlsanity.com/paper/2310.14005/) (2023)
- [DeepStratNet: A Context-Aware Coordinate Regression Framework for Seismic Horizon Tracking under Sparse Labels](https://mlsanity.com/paper/2610.02494/) (2026)

## Abstract (arXiv)

This paper presents a novel positive and negative set selection strategy for contrastive learning of medical images based on labels that can be extracted from clinical data. In the medical field, there exists a variety of labels for data that serve different purposes at different stages of a diagnostic and treatment process. Clinical labels and biomarker labels are two examples. In general, clinical labels are easier to obtain in larger quantities because they are regularly collected during routine clinical care, while biomarker labels require expert analysis and interpretation to obtain. Within the field of ophthalmology, previous work has shown that clinical values exhibit correlations with biomarker structures that manifest within optical coherence tomography (OCT) scans. We exploit this relationship by using the clinical data as pseudo-labels for our data without biomarker labels in order to choose positive and negative instances for training a backbone network with a supervised contrastive loss. In this way, a backbone network learns a representation space that aligns with the clinical data distribution available. Afterwards, we fine-tune the network trained in this manner with the smaller amount of biomarker labeled data with a cross-entropy loss in order to classify these key indicators of disease directly from OCT scans. We also expand on this concept by proposing a method that uses a linear combination of clinical contrastive losses. We benchmark our methods against state of the art self-supervised methods in a novel setting with biomarkers of varying granularity. We show performance improvements by as much as 5% in total biomarker detection AUROC.