mlsanityData to 14 Oct 2026

Audio MFCC-gram Transformers for respiratory insufficiency detection in COVID-19

arXiv 2210.14085 ↗First posted 25 Oct 2022cs.SD cs.LG eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

This work explores speech as a biomarker and investigates the detection of respiratory insufficiency (RI) by analyzing speech samples. Previous work citespira2021 constructed a dataset of respiratory insufficiency COVID-19 patient utterances and analyzed it by means of a convolutional neural network achieving an accuracy of 87.04%, validating the hypothesis that one can detect RI through speech. Here, we study how Transformer neural network architectures can improve the performance on RI detection. This approach enables construction of an acoustic model. By choosing the correct pretraining technique, we generate a self-supervised acoustic model, leading to improved performance (96.53%) of Transformers for RI detection.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 11 of its 24 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

11 of 24 references.

References · 11

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2016Layer Normalizationcited by 5,669
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  6. 2021AST: Audio Spectrogram Transformercited by 643
  7. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 340
  8. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 198
  9. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 188
  10. 2019Very Deep Self-Attention Networks for End-to-End Speech Recognitioncited by 46
  11. 2019Speech-XLNet: Unsupervised Acoustic Model Pretraining For Self-Attention Networkscited by 17

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 1; 2024: 2.

4 papers.

  1. 2022Pretrained audio neural networks for Speech emotion recognition in Portuguesecited by 3
  2. 2023Acoustic models of Brazilian Portuguese Speech based on Neural Transformerscited by 1
  3. 2024Discriminant audio properties in deep learning based respiratory insufficiency detection in Brazilian Portuguesecited by 1
  4. 2024Contrasting Deep Learning Models for Direct Respiratory Insufficiency Detection Versus Blood Oxygen Saturation Estimationcited by 1