# Deep Epidemiological Modeling by Black-box Knowledge Distillation: An Accurate Deep Learning Model for COVID-19

arXiv: https://arxiv.org/abs/2101.10280 · HTML: https://mlsanity.com/paper/2101.10280/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 20 Jan 2021.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 13 of 13)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [TinyBERT: Distilling BERT for Natural Language Understanding](https://mlsanity.com/paper/1909.10351/) (2019)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [Augmenting Data with Mixup for Sentence Classification: An Empirical Study](https://mlsanity.com/paper/1905.08941/) (2019)
- [Learning from Between-class Examples for Deep Sound Recognition](https://mlsanity.com/paper/1711.10282/) (2017)
- [Between-class Learning for Image Classification](https://mlsanity.com/paper/1711.10284/) (2017)
- [Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation from a Blackbox Model](https://mlsanity.com/paper/2003.13960/) (2020)
- [Accurate estimation of influenza epidemics using Google search data via ARGO](https://mlsanity.com/paper/1505.00864/) (2015)
- [Composite Monte Carlo Decision Making under High Uncertainty of Novel Coronavirus Epidemic Using Hybridized Deep Learning and Fuzzy Rule Induction](https://mlsanity.com/paper/2003.09868/) (2020)
- [Data-Efficient Ranking Distillation for Image Retrieval](https://mlsanity.com/paper/2007.05299/) (2020)

## Built on by (most cited first, 3 of 3)

- [ChatGPT for Shaping the Future of Dentistry: The Potential of Multi-Modal Large Language Model](https://mlsanity.com/paper/2304.03086/) (2023)
- [Pre-training Epidemic Time Series Forecasters with Compartmental Prototypes](https://mlsanity.com/paper/2502.03393/) (2025)
- [Epidemiology-informed Graph Neural Network for Heterogeneity-aware Epidemic Forecasting](https://mlsanity.com/paper/2411.17372/) (2024)

## Abstract (arXiv)

An accurate and efficient forecasting system is imperative to the prevention of emerging infectious diseases such as COVID-19 in public health. This system requires accurate transient modeling, lower computation cost, and fewer observation data. To tackle these three challenges, we propose a novel deep learning approach using black-box knowledge distillation for both accurate and efficient transmission dynamics prediction in a practical manner. First, we leverage mixture models to develop an accurate, comprehensive, yet impractical simulation system. Next, we use simulated observation sequences to query the simulation system to retrieve simulated projection sequences as knowledge. Then, with the obtained query data, sequence mixup is proposed to improve query efficiency, increase knowledge diversity, and boost distillation model accuracy. Finally, we train a student deep neural network with the retrieved and mixed observation-projection sequences for practical use. The case study on COVID-19 justifies that our approach accurately projects infections with much lower computation cost when observation data are limited.