# Speech-Based Estimation of Schizophrenia Severity Using Feature Fusion

arXiv: https://arxiv.org/abs/2411.06033 · HTML: https://mlsanity.com/paper/2411.06033/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2024.

## Built on by (most cited first, 3 of 3)

- [Multimodal Biomarkers for Schizophrenia: Towards Individual Symptom Severity Estimation](https://mlsanity.com/paper/2505.16044/) (2025)
- [Quantifying Articulatory Coordination as a Biomarker for Schizophrenia](https://mlsanity.com/paper/2511.03084/) (2025)
- [Articulation-Informed ASR: Integrating Articulatory Features into ASR via Auxiliary Speech Inversion and Cross-Attention Fusion](https://mlsanity.com/paper/2510.08585/) (2025)

## Abstract (arXiv)

Speech-based assessment of the schizophrenia spectrum has been widely researched over in the recent past. In this study, we develop a deep learning framework to estimate schizophrenia severity scores from speech using a feature fusion approach that fuses articulatory features with different self-supervised speech features extracted from pre-trained audio models. We also propose an auto-encoder-based self-supervised representation learning framework to extract compact articulatory embeddings from speech. Our top-performing speech-based fusion model with Multi-Head Attention (MHA) reduces Mean Absolute Error (MAE) by 9.18% and Root Mean Squared Error (RMSE) by 9.36% for schizophrenia severity estimation when compared with the previous models that combined speech and video inputs.