mlsanityData to 14 Oct 2026

BTS: Bridging Text and Sound Modalities for Metadata-Aided Respiratory Sound Classification

arXiv 2406.06786 ↗First posted 10 Jun 2024cs.SD cs.AI eess.AScited by 12

Not on the map. Why?

Abstract (arXiv)

Respiratory sound classification (RSC) is challenging due to varied acoustic signatures, primarily influenced by patient demographics and recording environments. To address this issue, we introduce a text-audio multimodal model that utilizes metadata of respiratory sounds, which provides useful complementary information for RSC. Specifically, we fine-tune a pretrained text-audio multimodal model using free-text descriptions derived from the sound samples' metadata which includes the gender and age of patients, type of recording devices, and recording location on the patient's body. Our method achieves state-of-the-art performance on the ICBHI dataset, surpassing the previous best result by a notable margin of 1.17%. This result validates the effectiveness of leveraging metadata and respiratory sound samples in enhancing RSC performance. Additionally, we investigate the model performance in the case where metadata is partially unavailable, which may occur in real-world clinical setting.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

16 of 25 references.

References · 16

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
  4. 2021AST: Audio Spectrogram Transformercited by 643
  5. 2021Open Vocabulary Electroencephalography-To-Text Decoding and Zero-shot Sentiment Classificationcited by 67
  6. 2020RespireNet: A Deep Neural Network for Accurately Detecting Abnormal Lung Sounds in Limited Data Settingcited by 26
  7. 2023Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classificationcited by 24
  8. 2021Prototype Learning for Interpretable Respiratory Sound Analysiscited by 21
  9. 2023Stethoscope-guided Supervised Contrastive Learning for Cross-domain Adaptation on Respiratory Sound Classificationcited by 16
  10. 2022Pretraining Respiratory Sound Representations using Metadata and Contrastive Learningcited by 15
  11. 2021Lung Sound Classification Using Co-tuning and Stochastic Normalizationcited by 14
  12. 2021Decoding EEG Brain Activity for Multi-Modal Natural Language Processingcited by 13
4 more
  1. 2022Example-based Explanations with Adversarial Attacks for Respiratory Sound Analysiscited by 13
  2. 2022Can Brain Signals Reveal Inner Alignment with Human Languages?cited by 12
  3. 2024RepAugment: Input-Agnostic Representation-Level Augmentation for Respiratory Sound Classificationcited by 12
  4. 2023Adversarial Fine-tuning using Generated Respiratory Sound to Address Class Imbalancecited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 5.

12 papers.

  1. 2024RespLLM: Unifying Audio and Text with Multimodal LLMs for Generalized Respiratory Health Predictioncited by 10
  2. 2025Tri-MTL: A Triple Multitask Learning Approach for Respiratory Disease Diagnosiscited by 3
  3. 2025Improving Respiratory Sound Classification with Architecture-Agnostic Knowledge Distillation from Ensemblescited by 3
  4. 2025Patient-Aware Feature Alignment for Robust Lung Sound Classification:Cohesion-Separation and Global Alignment Lossescited by 3
  5. 2024Lungmix: A Mixup-Based Strategy for Generalization in Respiratory Sound Classificationcited by 2
  6. 2026Mitigating Stethoscope-Induced Shortcuts in Respiratory Sound Classification under Federated Domain Generalization with Causality-Inspired Interventionscited by 1
  7. 2025Improving Deep Learning-based Respiratory Sound Analysis with Frequency Selection and Attention Mechanismcited by 0
  8. 2025Geometry-Aware Optimization for Respiratory Sound Classification: Enhancing Sensitivity with SAM-Optimized Audio Spectrogram Transformerscited by 0
  9. 2026Resp-Agent: An Agent-Based System for Multimodal Respiratory Sound Generation and Disease Diagnosiscited by 0
  10. 2026Meta-Ensemble Learning with Diverse Data Splits for Improved Respiratory Sound Classificationcited by 0
  11. 2026Quality Adaptive Angular Margin Learning for Respiratory Sound Classificationcited by 0
  12. 2026Lung-SRAD: Spectral-Aware Regularized Audio DASS with Dual-Axis Patch-Mix Contrastive Learning for Respiratory Sound Classificationcited by 0