# Artificial intelligence as a gateway to scientific discovery: Uncovering features in retinal fundus images

arXiv: https://arxiv.org/abs/2301.06675 · HTML: https://mlsanity.com/paper/2301.06675/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 41 references that are in the map.
- First posted on arXiv 17 Jan 2023.

## Builds on (most cited first, 11 of 11)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Deep Learning for Medical Image Analysis](https://mlsanity.com/paper/1708.08987/) (2017)
- [Understanding Neural Networks Through Deep Visualization](https://mlsanity.com/paper/1506.06579/) (2015)
- [Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning](https://mlsanity.com/paper/1708.09843/) (2017)
- [LadderNet: Multi-path networks based on U-Net for medical image segmentation](https://mlsanity.com/paper/1810.07810/) (2018)
- [DeepSeeNet: A deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs](https://mlsanity.com/paper/1811.07492/) (2018)
- [Learning from few examples: Classifying sex from retinal images via deep learning](https://mlsanity.com/paper/2207.09624/) (2022)

## Abstract (arXiv)

Purpose: Convolutional neural networks can be trained to detect various conditions or patient traits based on retinal fundus photographs, some of which, such as the patient sex, are invisible to the expert human eye. Here we propose a methodology for explainable classification of fundus images to uncover the mechanism(s) by which CNNs successfully predict the labels. We used patient sex as a case study to validate our proposed methodology. Approach: First, we used a set of 4746 fundus images, including training, validation and test partitions, to fine-tune a pre-trained CNN on the sex classification task. Next, we utilized deep learning explainability tools to hypothesize possible ways sex differences in the retina manifest. We measured numerous retinal properties relevant to our hypotheses through image segmentation to identify those significantly different between males and females. To tackle the multiple comparisons problem, we shortlisted the parameters by testing them on a set of 100 fundus images distinct from the images used for fine-tuning. Finally, we used an additional 400 images, not included in any previous set, to reveal significant sex differences in the retina. Results: We observed that the peripapillary area is darker in males compared to females (p=.023, d=.243). We also observed that males have richer retinal vasculature networks by showing a higher number of branches (p=.016, d=.272) and nodes (p=.014, d=.299) and a larger total length of branches (p=.045, d=.206) in the vessel graph. Also, vessels cover a greater area in the superior temporal quadrant of the retina in males compared to females (p=0.048, d=.194). Conclusions: Our methodology reveals retinal features in fundus photographs that allow CNNs to predict traits currently unknown, but meaningful to experts.