# Detecting Severity of Diabetic Retinopathy from Fundus Images: A Transformer Network-based Review

arXiv: https://arxiv.org/abs/2301.00973 · HTML: https://mlsanity.com/paper/2301.00973/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 62 references that are in the map.
- First posted on arXiv 3 Jan 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Going deeper with Image Transformers](https://mlsanity.com/paper/2103.17239/) (2021)
- [Metrics for Multi-Class Classification: an Overview](https://mlsanity.com/paper/2008.05756/) (2020)
- [Mish: A Self Regularized Non-Monotonic Activation Function](https://mlsanity.com/paper/1908.08681/) (2019)
- [Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy](https://mlsanity.com/paper/1710.01711/) (2017)
- [Zoom-in-Net: Deep Mining Lesions for Diabetic Retinopathy Detection](https://mlsanity.com/paper/1706.04372/) (2017)
- [Deep Learning based Computer-Aided Diagnosis Systems for Diabetic Retinopathy: A Survey](https://mlsanity.com/paper/1811.01238/) (2018)
- [SSiT: Saliency-guided Self-supervised Image Transformer for Diabetic Retinopathy Grading](https://mlsanity.com/paper/2210.10969/) (2022)
- [Search Algorithms for Automated Hyper-Parameter Tuning](https://mlsanity.com/paper/2104.14677/) (2021)
- [Deep Learning Approach to Diabetic Retinopathy Detection](https://mlsanity.com/paper/2003.02261/) (2020)
- [Multidimensional Contrast Limited Adaptive Histogram Equalization](https://mlsanity.com/paper/1906.11355/) (2019)

## Built on by (most cited first, 2 of 2)

- [A Comprehensive Review of Artificial Intelligence Applications in Major Retinal Conditions](https://mlsanity.com/paper/2311.13710/) (2023)
- [Efficient Deep Learning Approaches for Processing Ultra-Widefield Retinal Imaging](https://mlsanity.com/paper/2503.18151/) (2025)

## Abstract (arXiv)

Diabetic Retinopathy (DR) is considered one of the significant concerns worldwide, primarily due to its impact on causing vision loss among most people with diabetes. The severity of DR is typically comprehended manually by ophthalmologists from fundus photography-based retina images. This paper deals with an automated understanding of the severity stages of DR. In the literature, researchers have focused on this automation using traditional machine learning-based algorithms and convolutional architectures. However, the past works hardly focused on essential parts of the retinal image to improve the model performance. In this study, we adopt and fine-tune transformer-based learning models to capture the crucial features of retinal images for a more nuanced understanding of DR severity. Additionally, we explore the effectiveness of image transformers to infer the degree of DR severity from fundus photographs. For experiments, we utilized the publicly available APTOS-2019 blindness detection dataset, where the performances of the transformer-based models were quite encouraging.