# A Semi-Supervised Algorithm for Improving the Consistency of Crowdsourced Datasets: The COVID-19 Case Study on Respiratory Disorder Classification

arXiv: https://arxiv.org/abs/2209.04360 · HTML: https://mlsanity.com/paper/2209.04360/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 48 references that are in the map.
- First posted on arXiv 9 Sep 2022.

## Builds on (most cited first, 9 of 9)

- [Learning From Noisy Labels By Regularized Estimation Of Annotator Confusion](https://mlsanity.com/paper/1902.03680/) (2019)
- [Who Said What: Modeling Individual Labelers Improves Classification](https://mlsanity.com/paper/1703.08774/) (2017)
- [Direct Uncertainty Prediction for Medical Second Opinions](https://mlsanity.com/paper/1807.01771/) (2018)
- [The COUGHVID crowdsourcing dataset: A corpus for the study of large-scale cough analysis algorithms](https://mlsanity.com/paper/2009.11644/) (2020)
- [Coswara -- A Database of Breathing, Cough, and Voice Sounds for COVID-19 Diagnosis](https://mlsanity.com/paper/2005.10548/) (2020)
- [Exploring Self-Supervised Representation Ensembles for COVID-19 Cough Classification](https://mlsanity.com/paper/2105.07566/) (2021)
- [Detecting COVID-19 from Breathing and Coughing Sounds using Deep Neural Networks](https://mlsanity.com/paper/2012.14553/) (2020)
- [Assessment of Audio Features for Automatic Cough Detection](https://mlsanity.com/paper/2001.00580/) (2020)
- [Abductive reasoning as the basis to reproduce expert criteria in ECG Atrial Fibrillation identification](https://mlsanity.com/paper/1802.05998/) (2018)

## Built on by (most cited first, 7 of 7)

- [Robust COVID-19 Detection from Cough Sounds using Deep Neural Decision Tree and Forest: A Comprehensive Cross-Datasets Evaluation](https://mlsanity.com/paper/2501.01117/) (2025)
- [How to Count Coughs: An Event-Based Framework for Evaluating Automatic Cough Detection Algorithm Performance](https://mlsanity.com/paper/2406.01529/) (2024)
- [Sustaining model performance for covid-19 detection from dynamic audio data: Development and evaluation of a comprehensive drift-adaptive framework](https://mlsanity.com/paper/2409.19300/) (2024)
- [CoughViT: A Self-Supervised Vision Transformer for Cough Audio Representation Learning](https://mlsanity.com/paper/2508.03764/) (2025)
- [Sound Signal Synthesis with Auxiliary Classifier GAN, COVID-19 cough as an example](https://mlsanity.com/paper/2508.08892/) (2025)
- [Fine-tuning Pre-trained Audio Models for COVID-19 Detection: A Technical Report](https://mlsanity.com/paper/2511.14939/) (2025)
- [Why ML-based cough models do not generalize: a systematic cross-dataset evaluation for tuberculosis screening](https://mlsanity.com/paper/2608.25846/) (2026)

## Abstract (arXiv)

Cough audio signal classification is a potentially useful tool in screening for respiratory disorders, such as COVID-19. Since it is dangerous to collect data from patients with such contagious diseases, many research teams have turned to crowdsourcing to quickly gather cough sound data, as it was done to generate the COUGHVID dataset. The COUGHVID dataset enlisted expert physicians to diagnose the underlying diseases present in a limited number of uploaded recordings. However, this approach suffers from potential mislabeling of the coughs, as well as notable disagreement between experts. In this work, we use a semi-supervised learning (SSL) approach to improve the labeling consistency of the COUGHVID dataset and the robustness of COVID-19 versus healthy cough sound classification. First, we leverage existing SSL expert knowledge aggregation techniques to overcome the labeling inconsistencies and sparsity in the dataset. Next, our SSL approach is used to identify a subsample of re-labeled COUGHVID audio samples that can be used to train or augment future cough classification models. The consistency of the re-labeled data is demonstrated in that it exhibits a high degree of class separability, 3x higher than that of the user-labeled data, despite the expert label inconsistency present in the original dataset. Furthermore, the spectral differences in the user-labeled audio segments are amplified in the re-labeled data, resulting in significantly different power spectral densities between healthy and COVID-19 coughs, which demonstrates both the increased consistency of the new dataset and its explainability from an acoustic perspective. Finally, we demonstrate how the re-labeled dataset can be used to train a cough classifier. This SSL approach can be used to combine the medical knowledge of several experts to improve the database consistency for any diagnostic classification task.