# Uncertainty-Aware COVID-19 Detection from Imbalanced Sound Data

arXiv: https://arxiv.org/abs/2104.02005 · HTML: https://mlsanity.com/paper/2104.02005/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 33 references that are in the map.
- First posted on arXiv 5 Apr 2021.

## Builds on (most cited first, 19 of 19)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Bayesian Uncertainty Estimation for Batch Normalized Deep Networks](https://mlsanity.com/paper/1802.06455/) (2018)
- [Exploring Automatic Diagnosis of COVID-19 from Crowdsourced Respiratory Sound Data](https://mlsanity.com/paper/2006.05919/) (2020)
- [Coswara -- A Database of Breathing, Cough, and Voice Sounds for COVID-19 Diagnosis](https://mlsanity.com/paper/2005.10548/) (2020)
- [AI4COVID-19: AI Enabled Preliminary Diagnosis for COVID-19 from Cough Samples via an App](https://mlsanity.com/paper/2004.01275/) (2020)
- [Estimating Uncertainty and Interpretability in Deep Learning for Coronavirus (COVID-19) Detection](https://mlsanity.com/paper/2003.10769/) (2020)
- [COVID-19 Cough Classification using Machine Learning and Global Smartphone Recordings](https://mlsanity.com/paper/2012.01926/) (2020)
- [The INTERSPEECH 2021 Computational Paralinguistics Challenge: COVID-19 Cough, COVID-19 Speech, Escalation & Primates](https://mlsanity.com/paper/2102.13468/) (2021)
- [Cough Against COVID: Evidence of COVID-19 Signature in Cough Sounds](https://mlsanity.com/paper/2009.08790/) (2020)
- [Exploring Automatic COVID-19 Diagnosis via voice and symptoms from Crowdsourced Data](https://mlsanity.com/paper/2102.05225/) (2021)
- [COVID-19 and Computer Audition: An Overview on What Speech & Sound Analysis Could Contribute in the SARS-CoV-2 Corona Crisis](https://mlsanity.com/paper/2003.11117/) (2020)
- [Detecting COVID-19 from Breathing and Coughing Sounds using Deep Neural Networks](https://mlsanity.com/paper/2012.14553/) (2020)
- [An Early Study on Intelligent Analysis of Speech under COVID-19: Severity, Sleep Quality, Fatigue, and Anxiety](https://mlsanity.com/paper/2005.00096/) (2020)
- [A Real-time Robot-based Auxiliary System for Risk Evaluation of COVID-19 Infection](https://mlsanity.com/paper/2008.07695/) (2020)
- [The Benefit of the Doubt: Uncertainty Aware Sensing for Edge Computing Platforms](https://mlsanity.com/paper/2102.05956/) (2021)

## Built on by (most cited first, 6 of 6)

- [Data-IQ: Characterizing subgroups with heterogeneous outcomes in tabular data](https://mlsanity.com/paper/2210.13043/) (2022)
- [Sounds of COVID-19: exploring realistic performance of audio-based digital testing](https://mlsanity.com/paper/2106.15523/) (2021)
- [Benchmarking Uncertainty Quantification on Biosignal Classification Tasks under Dataset Shift](https://mlsanity.com/paper/2112.09196/) (2021)
- [A Summary of the ComParE COVID-19 Challenges](https://mlsanity.com/paper/2202.08981/) (2022)
- [High Frequency EEG Artifact Detection with Uncertainty via Early Exit Paradigm](https://mlsanity.com/paper/2107.10746/) (2021)
- [UFRC: A Unified Framework for Reliable COVID-19 Detection on Crowdsourced Cough Audio](https://mlsanity.com/paper/2204.07763/) (2022)

## Abstract (arXiv)

Recently, sound-based COVID-19 detection studies have shown great promise to achieve scalable and prompt digital pre-screening. However, there are still two unsolved issues hindering the practice. First, collected datasets for model training are often imbalanced, with a considerably smaller proportion of users tested positive, making it harder to learn representative and robust features. Second, deep learning models are generally overconfident in their predictions. Clinically, false predictions aggravate healthcare costs. Estimation of the uncertainty of screening would aid this. To handle these issues, we propose an ensemble framework where multiple deep learning models for sound-based COVID-19 detection are developed from different but balanced subsets from original data. As such, data are utilized more effectively compared to traditional up-sampling and down-sampling approaches: an AUC of 0.74 with a sensitivity of 0.68 and a specificity of 0.69 is achieved. Simultaneously, we estimate uncertainty from the disagreement across multiple models. It is shown that false predictions often yield higher uncertainty, enabling us to suggest the users with certainty higher than a threshold to repeat the audio test on their phones or to take clinical tests if digital diagnosis still fails. This study paves the way for a more robust sound-based COVID-19 automated screening system.