# Zero-Shot Respiratory Sound Classification through LLM-Augmented Audio-Text Alignment

arXiv: https://arxiv.org/abs/2609.00055 · HTML: https://mlsanity.com/paper/2609.00055/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 30 Aug 2026.

## Builds on (most cited first, 17 of 17)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution](https://mlsanity.com/paper/2202.10054/) (2022)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models](https://mlsanity.com/paper/2507.08128/) (2025)
- [The COUGHVID crowdsourcing dataset: A corpus for the study of large-scale cough analysis algorithms](https://mlsanity.com/paper/2009.11644/) (2020)
- [Understanding Multimodal Contrastive Learning and Incorporating Unpaired Data](https://mlsanity.com/paper/2302.06232/) (2023)
- [Towards Open Respiratory Acoustic Foundation Models: Pretraining and Benchmarking](https://mlsanity.com/paper/2406.16148/) (2024)
- [Coswara: A respiratory sounds and symptoms dataset for remote screening of SARS-CoV-2 infection](https://mlsanity.com/paper/2305.12741/) (2023)
- [Audio-based AI classifiers show no evidence of improved COVID-19 screening over simple symptoms checkers](https://mlsanity.com/paper/2212.08570/) (2022)
- [RespLLM: Unifying Audio and Text with Multimodal LLMs for Generalized Respiratory Health Prediction](https://mlsanity.com/paper/2410.05361/) (2024)
- [Language-Image Alignment with Fixed Text Encoders](https://mlsanity.com/paper/2506.04209/) (2025)
- [Towards Pre-training an Effective Respiratory Audio Foundation Model](https://mlsanity.com/paper/2505.15307/) (2025)

## Abstract (arXiv)

Self-supervised respiratory encoders lack semantic grounding in clinical domain needed for zero-shot inference, limiting their utility without task-specific labeled data. We propose a framework that aligns these encoders with medical terminology in a shared latent space turning them into a zero-shot-capable foundation model. To address paired data scarcity, we use a medical LLM to synthesize structured reports from metadata, creating dense semantic anchors for contrastive learning. Our training combines a sigmoid-based contrastive loss with encoder's native SSL objective and similarity-aware negative sampling to sharpen pathological boundaries. Across 9 tasks on 6 datasets, our method achieves a 61.3% mean zero-shot AUC, surpassing CLAP (51.4%) and Qwen2-Audio (54.9%) while reaching the highest linear probing AUC (71.6%) with only 43% of data used by full-scale baselines, showing that structured semantic alignment outperforms large-scale, general-purpose models in clinical diagnostics.