mlsanityData to 14 Oct 2026

KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification

arXiv 2512.09069 ↗First posted 9 Dec 2025cs.CV cs.AI cs.LGcited by 0

On the map: Knowledge distillation → Distillation for medical imaging

Abstract (arXiv)

Age-related macular degeneration (AMD) and choroidal neovascularization (CNV)-related conditions are leading causes of vision loss worldwide, with optical coherence tomography (OCT) serving as a cornerstone for early detection and management. However, deploying state-of-the-art deep learning models like ConvNeXtV2-Large in clinical settings is hindered by their computational demands. Therefore, it is desirable to develop efficient models that maintain high diagnostic performance while enabling real-time deployment. In this study, a novel knowledge distillation framework, termed KD-OCT, is proposed to compress a high-performance ConvNeXtV2-Large teacher model, enhanced with advanced augmentations, stochastic weight averaging, and focal loss, into a lightweight EfficientNet-B2 student for classifying normal, drusen, and CNV cases. KD-OCT employs real-time distillation with a combined loss balancing soft teacher knowledge transfer and hard ground-truth supervision. The effectiveness of the proposed method is evaluated on the Noor Eye Hospital (NEH) dataset using patient-level cross-validation. Experimental results demonstrate that KD-OCT outperforms comparable multi-scale or feature-fusion OCT classifiers in efficiency-accuracy balance, achieving near-teacher performance with substantial reductions in model size and inference time. Despite the compression, the student model exceeds most existing frameworks, facilitating edge deployment for AMD screening. Code is available at https://github.com/erfan-nourbakhsh/KD-OCT.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Distillation for medical imaging, one of 12 directions in Knowledge distillation. Builds on 18 of its 54 references that are in the map. First posted on arXiv 9 Dec 2025.

Builds on

18 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Cross-Architecture Knowledge Distillation (KD) for Retinal Fundus Image Anomaly Detection on NVIDIA Jetson Nano23 Jun

Other references · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2016Densely Connected Convolutional Networkscited by 6,720
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  9. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  10. 2017Squeeze-and-Excitation Networkscited by 4,789
  11. 2022A ConvNet for the 2020scited by 3,579
  12. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
5 more
  1. 2025MedGemma Technical Reportcited by 436
  2. 2018Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networkscited by 280
  3. 2021Multi-Scale Convolutional Neural Network for Automated AMD Classification using Retinal OCT Imagescited by 10
  4. 2024ELA: Efficient Local Attention for Deep Convolutional Neural Networkscited by 8
  5. 2023Fundus-Enhanced Disease-Aware Distillation Model for Retinal Disease Classification from OCT Imagescited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Can Multimodal Large Language Models Understand OCT?
  2. 2023IKD+: Reliable Low Complexity Deep Models For Retinopathy Classification
  3. 2024MultiEYE: Dataset and Benchmark for OCT-Enhanced Retinal Disease Recognition from Fundus Images
  4. 2022Feature Representation Learning for Robust Retinal Disease Detection from Optical Coherence Tomography Images
  5. 2023Leveraging Complementary Attention maps in vision transformers for OCT image analysis
  6. 2023Retinal OCT Synthesis with Denoising Diffusion Probabilistic Models for Layer Segmentation
  7. 2021Relational Subsets Knowledge Distillation for Long-tailed Retinal Diseases Recognition
  8. 2023Reverse Knowledge Distillation: Training a Large Model using a Small One for Retinal Image Matching on Limited Data