# Fused Audio Instance and Representation for Respiratory Disease Detection

arXiv: https://arxiv.org/abs/2204.10581 · HTML: https://mlsanity.com/paper/2204.10581/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 39 references that are in the map.
- First posted on arXiv 22 Apr 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Exploring Automatic Diagnosis of COVID-19 from Crowdsourced Respiratory Sound Data](https://mlsanity.com/paper/2006.05919/) (2020)
- [The COUGHVID crowdsourcing dataset: A corpus for the study of large-scale cough analysis algorithms](https://mlsanity.com/paper/2009.11644/) (2020)
- [Coswara -- A Database of Breathing, Cough, and Voice Sounds for COVID-19 Diagnosis](https://mlsanity.com/paper/2005.10548/) (2020)
- [COVID-19 Cough Classification using Machine Learning and Global Smartphone Recordings](https://mlsanity.com/paper/2012.01926/) (2020)
- [How Transferable Are Self-supervised Features in Medical Image Classification Tasks?](https://mlsanity.com/paper/2108.10048/) (2021)
- [DiCOVA Challenge: Dataset, task, and baseline system for COVID-19 diagnosis using acoustics](https://mlsanity.com/paper/2103.09148/) (2021)
- [Automatic Cough Classification for Tuberculosis Screening in a Real-World Environment](https://mlsanity.com/paper/2103.13300/) (2021)
- [Exploring Self-Supervised Representation Ensembles for COVID-19 Cough Classification](https://mlsanity.com/paper/2105.07566/) (2021)
- [Detection of COVID-19 through the analysis of vocal fold oscillations](https://mlsanity.com/paper/2010.10707/) (2020)
- [Virufy: A Multi-Branch Deep Learning Network for Automated Detection of COVID-19](https://mlsanity.com/paper/2103.01806/) (2021)
- [Multimedia Respiratory Database (RespiratoryDatabase@TR): Auscultation Sounds and Chest X-rays](https://mlsanity.com/paper/2101.10946/) (2021)
- [Audio feature ranking for sound-based COVID-19 patient detection](https://mlsanity.com/paper/2104.07128/) (2021)

## Abstract (arXiv)

Audio-based classification techniques on body sounds have long been studied to aid in the diagnosis of respiratory diseases. While most research is centered on the use of cough as the main biomarker, other body sounds also have the potential to detect respiratory diseases. Recent studies on COVID-19 have shown that breath and speech sounds, in addition to cough, correlate with the disease. Our study proposes Fused Audio Instance and Representation (FAIR) as a method for respiratory disease detection. FAIR relies on constructing a joint feature vector from various body sounds represented in waveform and spectrogram form. We conducted experiments on the use case of COVID-19 detection by combining waveform and spectrogram representation of body sounds. Our findings show that the use of self-attention to combine extracted features from cough, breath, and speech sounds leads to the best performance with an Area Under the Receiver Operating Characteristic Curve (AUC) score of 0.8658, a sensitivity of 0.8057, and a specificity of 0.7958. Compared to models trained solely on spectrograms or waveforms, the use of both representations results in an improved AUC score, demonstrating that combining spectrogram and waveform representation helps to enrich the extracted features and outperforms the models that use only one representation.