# A distillation based approach for the diagnosis of diseases

arXiv: https://arxiv.org/abs/2108.03470 · HTML: https://mlsanity.com/paper/2108.03470/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 7 Aug 2021.

## Builds on (most cited first, 13 of 13)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Improved Knowledge Distillation via Teacher Assistant](https://mlsanity.com/paper/1902.03393/) (2019)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [On the Efficacy of Knowledge Distillation](https://mlsanity.com/paper/1910.01348/) (2019)
- [Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks](https://mlsanity.com/paper/2004.05937/) (2020)
- [Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding](https://mlsanity.com/paper/1904.09482/) (2019)
- [Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification](https://mlsanity.com/paper/1801.09927/) (2018)
- [Large Scale Automated Reading of Frontal and Lateral Chest X-Rays using Dual Convolutional Neural Networks](https://mlsanity.com/paper/1804.07839/) (2018)
- [Iterative Attention Mining for Weakly Supervised Thoracic Disease Pattern Localization in Chest X-Rays](https://mlsanity.com/paper/1807.00958/) (2018)
- [Automated Methods for Detection and Classification Pneumonia based on X-Ray Images Using Deep Learning](https://mlsanity.com/paper/2003.14363/) (2020)

## Abstract (arXiv)

Presently, Covid-19 is a serious threat to the world at large. Efforts are being made to reduce disease screening times and in the development of a vaccine to resist this disease, even as thousands succumb to it everyday. We propose a novel method of automated screening of diseases like Covid-19 and pneumonia from Chest X-Ray images with the help of Computer Vision. Unlike computer vision classification algorithms which come with heavy computational costs, we propose a knowledge distillation based approach which allows us to bring down the model depth, while preserving the accuracy. We make use of an augmentation of the standard distillation module with an auxiliary intermediate assistant network that aids in the continuity of the flow of information. Following this approach, we are able to build an extremely light student network, consisting of just 3 convolutional blocks without any compromise on accuracy. We thus propose a method of classification of diseases which can not only lead to faster screening, but can also operate seamlessly on low-end devices.