# Multi-View Speech Representation Learning for Parkinson's Disease Detection Using Context-guided Cross-modal Attention

arXiv: https://arxiv.org/abs/2606.09271 · HTML: https://mlsanity.com/paper/2606.09271/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 38 references that are in the map.
- First posted on arXiv 8 Jun 2026.

## Builds on (most cited first, 6 of 6)

- [Beyond Bilinear: Generalized Multimodal Factorized High-order Pooling for Visual Question Answering](https://mlsanity.com/paper/1708.03619/) (2017)
- [Exploiting Foundation Models and Speech Enhancement for Parkinson's Disease Detection from Speech in Real-World Operative Conditions](https://mlsanity.com/paper/2406.16128/) (2024)
- [Unveiling Interpretability in Self-Supervised Speech Representations for Parkinson's Diagnosis](https://mlsanity.com/paper/2412.02006/) (2024)
- [Bilingual Dual-Head Deep Model for Parkinson's Disease Detection from Speech](https://mlsanity.com/paper/2503.10301/) (2025)
- [Supervised Speech Representation Learning for Parkinson's Disease Classification](https://mlsanity.com/paper/2106.00531/) (2021)
- [Evaluating the Usefulness of Non-Diagnostic Speech Data for Developing Parkinson's Disease Classifiers](https://mlsanity.com/paper/2505.18722/) (2025)

## Abstract (arXiv)

Parkinson's disease (PD) is a progressive neurodegenerative disorder that frequently causes speech impairments associated with hypokinetic dysarthria. As speech production relies on the precise coordination of complex neuromuscular mechanisms, speech analysis has emerged as a promising non-invasive and cost-effective biomarker for early PD detection. Recent deep learning approaches have shown encouraging results; however, most existing methods rely on a single speech representation, potentially overlooking complementary pathological information encoded across different feature spaces. In this work, we propose a multi-branch deep learning framework for automatic PD detection from speech. Each recording is segmented into 5-second chunks and represented using three complementary modalities: Log-Mel spectrograms, MFCCs, and HuBERT embeddings extracted from raw waveforms. The spectrograms are processed using a pre-trained ResNet-18 encoder, MFCC sequences are modeled through a BiLSTM network, and raw speech is encoded using a pre-trained HuBERT model. To effectively integrate these heterogeneous representations, we introduce a context-guided cross-modal attention mechanism that dynamically weights temporal HuBERT embeddings according to the global acoustic context derived from the spectrogram and MFCC branches. Experiments conducted on the publicly available Spanish PC-GITA corpus under strict speaker-independent 5-fold cross-validation demonstrate the effectiveness of the proposed approach. The proposed architecture achieves an accuracy of 91.51%, an F1-score of 91.24%, and an AUC of 95.97%. Furthermore, ablation studies confirm the contribution of both the proposed context-guided cross-modal attention mechanism and the integration of complementary speech representations. These findings highlight the potential of heterogeneous speech modeling for robust and clinically reliable PD detection.