mlsanityData to 14 Oct 2026

OLIVES Dataset: Ophthalmic Labels for Investigating Visual Eye Semantics

arXiv 2209.11195 ↗First posted 22 Sep 2022eess.IV cs.CV cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

Clinical diagnosis of the eye is performed over multifarious data modalities including scalar clinical labels, vectorized biomarkers, two-dimensional fundus images, and three-dimensional Optical Coherence Tomography (OCT) scans. Clinical practitioners use all available data modalities for diagnosing and treating eye diseases like Diabetic Retinopathy (DR) or Diabetic Macular Edema (DME). Enabling usage of machine learning algorithms within the ophthalmic medical domain requires research into the relationships and interactions between all relevant data over a treatment period. Existing datasets are limited in that they neither provide data nor consider the explicit relationship modeling between the data modalities. In this paper, we introduce the Ophthalmic Labels for Investigating Visual Eye Semantics (OLIVES) dataset that addresses the above limitation. This is the first OCT and near-IR fundus dataset that includes clinical labels, biomarker labels, disease labels, and time-series patient treatment information from associated clinical trials. The dataset consists of 1268 near-IR fundus images each with at least 49 OCT scans, and 16 biomarkers, along with 4 clinical labels and a disease diagnosis of DR or DME. In total, there are 96 eyes' data averaged over a period of at least two years with each eye treated for an average of 66 weeks and 7 injections. We benchmark the utility of OLIVES dataset for ophthalmic data as well as provide benchmarks and concrete research directions for core and emerging machine learning paradigms within medical image analysis.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 29 of its 86 references that are in the map. First posted on arXiv 22 Sep 2022.

Builds on

29 of 86 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  6. 2020Supervised Contrastive Learningcited by 2,829
  7. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  8. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
  9. 2020The Future of Digital Health with Federated Learningcited by 633
  10. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  11. 2020Ethical Machine Learning in Health Carecited by 113
  12. 2019Attention Based Glaucoma Detection: A Large-scale Database and CNN Modelcited by 76
17 more
  1. 2014Visual Causal Feature Learningcited by 59
  2. 2021A Benchmark of Ocular Disease Intelligent Recognition: One Shot for Multi-disease Detectioncited by 34
  3. 2017CURE-TSR: Challenging Unreal and Real Environments for Traffic Sign Recognitioncited by 32
  4. 2019A Machine Learning Benchmark for Facies Classificationcited by 27
  5. 2020Contrastive Explanations in Neural Networkscited by 26
  6. 2019Modeling Disease Progression In Retinal OCTs With Longitudinal Self-Supervised Learningcited by 22
  7. 2021Action Segmentation with Mixed Temporal Domain Adaptationcited by 21
  8. 2018CURE-OR: Challenging Unreal and Real Environments for Object Recognitioncited by 19
  9. 2018A feature agnostic approach for glaucoma detection in OCT volumescited by 12
  10. 2021Extracting Causal Visual Features for Limited label Classificationcited by 12
  11. 2022Volumetric Supervised Contrastive Learning for Seismic Semantic Segmentationcited by 12
  12. 2021Contrastive Reasoning in Neural Networkscited by 11
  13. 2022Patient Aware Active Learning for Fine-Grained OCT Classificationcited by 9
  14. 2019Relative Afferent Pupillary Defect Screening through Transfer Learningcited by 8
  15. 2022Multi-Modal Learning Using Physicians Diagnostics for Optical Coherence Tomography Classificationcited by 7
  16. 2022DECAL: DEployable Clinical Active Learningcited by 7
  17. 2020Fabric Surface Characterization: Assessment of Deep Learning-based Texture Representations Using a Challenging Datasetcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 5; 2024: 9; 2025: 5; 2026: 3.

24 papers.

Sort:

By area:

  1. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 184
  2. 2024A review of deep learning-based information fusion techniques for multimodal medical image classificationcited by 25
  3. 2023Clinically Labeled Contrastive Learning for OCT Biomarker Classificationcited by 9
  4. 2024Improving Representation of High-frequency Components for Medical Visual Foundation Modelscited by 9
  5. 2025Hierarchical and Multimodal Data for Daily Activity Understandingcited by 7
  6. 2025MIRAGE: Multimodal foundation model and benchmark for comprehensive retinal OCT image analysiscited by 7
  7. 2024Confidence-aware multi-modality learning for eye disease screeningcited by 5
  8. 2024F3OCUS -- Federated Finetuning of Vision-Language Foundation Models with Optimal Client Layer Updating Strategy via Multi-objective Meta-Heuristicscited by 4
  9. 2025A Large-scale Benchmark on Geological Fault Delineation Models: Domain Shift, Training Dynamics, Generalizability, Evaluation and Inferential Behaviorcited by 4
  10. 2022On the Ramifications of Human Label Uncertaintycited by 3
  11. 2023Clinical Trial Active Learningcited by 3
  12. 2024Taxes Are All You Need: Integration of Taxonomical Hierarchy Relationships into the Contrastive Losscited by 3
12 more
  1. 2022Clinical Contrastive Learning for Biomarker Detectioncited by 2
  2. 2023Stochastic Surprisal: An inferential measurement of Free Energy in Neural Networkscited by 2
  3. 2023Transformer-based end-to-end classification of variable-length volumetric datacited by 2
  4. 2024Are Objective Explanatory Evaluation metrics Trustworthy? An Adversarial Analysiscited by 2
  5. 2024CRACKS: Crowdsourcing Resources for Analysis and Categorization of Key Subsurface faultscited by 2
  6. 2026Project Imaging-X: A Survey of 1000+ Open-Access Medical Imaging Datasets for Foundation Model Developmentcited by 2
  7. 2024Visual Question Answering in Ophthalmology: A Progressive and Practical Perspectivecited by 1
  8. 2025Countering Multi-modal Representation Collapse through Rank-targeted Fusioncited by 1
  9. 2023Leveraging Complementary Attention maps in vision transformers for OCT image analysiscited by 0
  10. 2025Code and Pixels: Multi-Modal Contrastive Pre-training for Enhanced Tabular Data Analysiscited by 0
  11. 2026Automated 2D and 3D Segmentation of AMD and DME Lesions in OCTcited by 0
  12. 2026SynCo: Learning Cross-Modal Synergy by Contrasting Interaction Residualscited by 0