mlsanityData to 14 Oct 2026

EyeMVP: OCT-Informed Fundus Representation Learning via Paired CFP--OCT Pretraining

arXiv 2606.15129 ↗First posted 13 Jun 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Color fundus photography (CFP) is the mainstay of large-scale retinal screening, but its diagnostic capacity is limited by the lack of depth-resolved structure, which optical coherence tomography (OCT) provides yet is less accessible at population scale. We present EyeMVP, a cross-modal retinal foundation model that uses paired CFP--OCT pretraining to learn OCT-informed CFP representations while requiring only CFP at inference. Pretrained on 674,893 same-eye same-day CFP--OCT triples from 112,642 patients across eight hospitals, EyeMVP uses cross-modal masked reconstruction to enrich CFP features with OCT-associated supervision, and combines source-constrained cross-attention with CFP-derived structural masks to accommodate the non-aligned geometry of en-face CFP and cross-sectional OCT. Across 15 dataset-level settings spanning classification and segmentation, under both full-data and few-shot regimes, EyeMVP performs on par with or better than representative retinal foundation models, with consistent gains on macular and optic-nerve tasks; it attains AUROCs of 0.923 for macular edema and 0.867 for myopic macular schisis, two conditions poorly resolved in CFP. In an exploratory reader study, EyeMVP surpasses junior and intermediate ophthalmologists but not seniors on macular edema, while exceeding all groups on myopic macular schisis. These results indicate that cross-modal reconstruction can enrich CFP representations with OCT-associated supervision, offering a practical route to stronger CFP-based screening.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 42 references that are in the map. First posted on arXiv 13 Jun 2026.

Builds on

18 of 42 references.

References · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2022A ConvNet for the 2020scited by 3,579
  6. 2018Attention U-Net: Learning Where to Look for the Pancreascited by 1,435
  7. 2019UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentationcited by 586
  8. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  9. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 291
  10. 2019REFUGE Challenge: A Unified Framework for Evaluating Automated Methods for Glaucoma Assessment from Fundus Photographscited by 228
  11. 2023BioCLIP: A Vision Foundation Model for the Tree of Lifecited by 161
  12. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 148
6 more
  1. 2022Multi-Modal Masked Autoencoders for Medical Vision-and-Language Pre-Trainingcited by 104
  2. 2023A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervisioncited by 78
  3. 2024EyeFound: A Multimodal Generalist Foundation Model for Ophthalmic Imagingcited by 18
  4. 2018From Machine to Machine: An OCT-trained Deep Learning Algorithm for Objective Quantification of Glaucomatous Damage in Fundus Photographscited by 9
  5. 2024MultiEYE: Dataset and Benchmark for OCT-Enhanced Retinal Disease Recognition from Fundus Imagescited by 9
  6. 2025MIRAGE: Multimodal foundation model and benchmark for comprehensive retinal OCT image analysiscited by 7

Built on by

Not cited yet.