# Explainable Multi-Modal Deep Learning for Automatic Detection of Lung Diseases from Respiratory Audio Signals

arXiv: https://arxiv.org/abs/2512.00563 · HTML: https://mlsanity.com/paper/2512.00563/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 36 references that are in the map.
- First posted on arXiv 29 Nov 2025.

## Builds on (most cited first, 5 of 5)

- [Deep auscultation: Predicting respiratory anomalies and diseases via recurrent neural networks](https://mlsanity.com/paper/1907.05708/) (2019)
- [Robust Deep Learning Framework For Predicting Respiratory Anomalies and Diseases](https://mlsanity.com/paper/2002.03894/) (2020)
- [Robust and Interpretable Temporal Convolution Network for Event Detection in Lung Sound Recordings](https://mlsanity.com/paper/2106.15835/) (2021)
- [Towards reliable respiratory disease diagnosis based on cough sounds and vision transformers](https://mlsanity.com/paper/2408.15667/) (2024)
- [Enhanced SegNet with Integrated Grad-CAM for Interpretable Retinal Layer Segmentation in OCT Images](https://mlsanity.com/paper/2509.07795/) (2025)

## Abstract (arXiv)

Respiratory diseases remain major global health challenges, and traditional auscultation is often limited by subjectivity, environmental noise, and inter-clinician variability. This study presents an explainable multimodal deep learning framework for automatic lung-disease detection using respiratory audio signals. The proposed system integrates two complementary representations: a spectral-temporal encoder based on a CNN-BiLSTM Attention architecture, and a handcrafted acoustic-feature encoder capturing physiologically meaningful descriptors such as MFCCs, spectral centroid, spectral bandwidth, and zero-crossing rate. These branches are combined through late-stage fusion to leverage both data-driven learning and domain-informed acoustic cues. The model is trained and evaluated on the Asthma Detection Dataset Version 2 using rigorous preprocessing, including resampling, normalization, noise filtering, data augmentation, and patient-level stratified partitioning. The study achieved strong generalization with 91.21% accuracy, 0.899 macro F1-score, and 0.9866 macro ROC-AUC, outperforming all ablated variants. An ablation study confirms the importance of temporal modeling, attention mechanisms, and multimodal fusion. The framework incorporates Grad-CAM, Integrated Gradients, and SHAP, generating interpretable spectral, temporal, and feature-level explanations aligned with known acoustic biomarkers to build clinical transparency. The findings demonstrate the framework's potential for telemedicine, point-of-care diagnostics, and real-world respiratory screening.