# IKD+: Reliable Low Complexity Deep Models For Retinopathy Classification

arXiv: https://arxiv.org/abs/2303.02310 · HTML: https://mlsanity.com/paper/2303.02310/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 22 references that are in the map.
- First posted on arXiv 4 Mar 2023.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Multivariate Confidence Calibration for Object Detection](https://mlsanity.com/paper/2004.13546/) (2020)
- [Deep Learning based Computer-Aided Diagnosis Systems for Diabetic Retinopathy: A Survey](https://mlsanity.com/paper/1811.01238/) (2018)
- [Calibrating Deep Neural Networks using Explicit Regularisation and Dynamic Data Pruning](https://mlsanity.com/paper/2212.10005/) (2022)
- [Multi-Disease Detection in Retinal Imaging based on Ensembling Heterogeneous Deep Learning Models](https://mlsanity.com/paper/2103.14660/) (2021)

## Built on by (most cited first, 3 of 3)

- [Region Guided Attention Network for Retinal Vessel Segmentation](https://mlsanity.com/paper/2407.18970/) (2024)
- [LFRA-Net: A Lightweight Focal and Region-Aware Attention Network for Retinal Vessel Segmentatio](https://mlsanity.com/paper/2509.11811/) (2025)
- [EDDense-Net: Fully Dense Encoder Decoder Network for Joint Segmentation of Optic Cup and Disc](https://mlsanity.com/paper/2308.10192/) (2023)

## Abstract (arXiv)

Deep neural network (DNN) models for retinopathy have estimated predictive accuracies in the mid-to-high 90%. However, the following aspects remain unaddressed: State-of-the-art models are complex and require substantial computational infrastructure to train and deploy; The reliability of predictions can vary widely. In this paper, we focus on these aspects and propose a form of iterative knowledge distillation(IKD), called IKD+ that incorporates a tradeoff between size, accuracy and reliability. We investigate the functioning of IKD+ using two widely used techniques for estimating model calibration (Platt-scaling and temperature-scaling), using the best-performing model available, which is an ensemble of EfficientNets with approximately 100M parameters. We demonstrate that IKD+ equipped with temperature-scaling results in models that show up to approximately 500-fold decreases in the number of parameters than the original ensemble without a significant loss in accuracy. In addition, calibration scores (reliability) for the IKD+ models are as good as or better than the base mode