# Training a high-performance retinal foundation model with half-the-data and 400 times less compute

arXiv: https://arxiv.org/abs/2405.00117 · HTML: https://mlsanity.com/paper/2405.00117/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 60 references that are in the map.
- First posted on arXiv 30 Apr 2024.

## Builds on (most cited first, 19 of 19)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles](https://mlsanity.com/paper/2306.00989/) (2023)
- [Fixing the train-test resolution discrepancy](https://mlsanity.com/paper/1906.06423/) (2019)
- [A Closer Look at AUROC and AUPRC under Class Imbalance](https://mlsanity.com/paper/2401.06091/) (2024)
- [Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy](https://mlsanity.com/paper/1710.01711/) (2017)
- [AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge](https://mlsanity.com/paper/2302.01738/) (2023)
- [GPViT: A High Resolution Non-Hierarchical Vision Transformer with Group Propagation](https://mlsanity.com/paper/2212.06795/) (2022)

## Built on by (most cited first, 9 of 9)

- [Foundation Models in Medical Imaging: A Review and Outlook](https://mlsanity.com/paper/2506.09095/) (2025)
- [Managing Diabetic Retinopathy with Deep Learning: A Data Centric Overview](https://mlsanity.com/paper/2604.02448/) (2026)
- [A Pre-trained Foundation Model Framework for Multiplanar MRI Classification of Extramural Vascular Invasion and Mesorectal Fascia Invasion in Rectal Cancer](https://mlsanity.com/paper/2505.18058/) (2025)
- [Prompt Mechanisms in Medical Imaging: A Comprehensive Survey](https://mlsanity.com/paper/2507.01055/) (2025)
- [ALL-PET: A Low-resource and Low-shot PET Foundation Model in Projection Domain](https://mlsanity.com/paper/2509.09130/) (2025)
- [Fair-Eye Net: A Fair, Trustworthy, Multimodal Integrated Glaucoma Full Chain AI System](https://mlsanity.com/paper/2601.18464/) (2026)
- [Towards Interpretable Foundation Models for Retinal Fundus Images](https://mlsanity.com/paper/2603.18846/) (2026)
- [Representation Transfer of Foundation Models for Ultra-Widefield Retinal Imaging](https://mlsanity.com/paper/2608.00586/) (2026)
- [Beyond Natural-Image Foundation Models: Benchmarking Satellite Pretraining for Ophthalmic Image Analysis](https://mlsanity.com/paper/2608.15195/) (2026)

## Abstract (arXiv)

Artificial Intelligence in medicine is traditionally limited by the lack of massive training datasets. Foundation models, pre-trained models that can be adapted to downstream tasks with small datasets, could alleviate this problem. Researchers at Moorfields Eye Hospital (MEH) proposed RETFound-MEH, a retinal foundation model trained on 900,000 images, including private hospital data. Recently, data-efficient DERETFound was proposed providing comparable performance while being trained on only 150,000 publicly available images. However, both these models required very substantial resources to train initially and are resource-intensive in downstream use. We propose a novel Token Reconstruction objective that we use to train RETFound-Green, a retinal foundation model trained using only 75,000 publicly available images and 400 times less compute. We estimate the cost of training RETFound-MEH and DERETFound at 10,000 and 14,000, respectively. RETFound-Green could be trained for less than 100, with equally reduced environmental impact. RETFound-Green is also far more efficient in downstream use: it can be downloaded 14 times faster, computes vector embeddings 2.7 times faster which then require 2.6 times less storage space. Despite this, RETFound-Green does not perform systematically worse. In fact, on various task on three downstream datasets from Brazil, India and China, it performs best on 68 tasks out of 119 comparisons, versus 21 for DERETFound and 13 for RETFound-MEH. Our results suggest that RETFound-Green is a very efficient, high-performance retinal foundation model. We anticipate that our Token Reconstruction objective could be scaled up for even higher performance and be applied to other domains beyond retinal imaging.