# Multimodal Biomarkers for Schizophrenia: Towards Individual Symptom Severity Estimation

arXiv: https://arxiv.org/abs/2505.16044 · HTML: https://mlsanity.com/paper/2505.16044/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 21 May 2025.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Improving Speech Inversion Through Self-Supervised Embeddings and Enhanced Tract Variables](https://mlsanity.com/paper/2309.09220/) (2023)
- [Multimodal Approach for Assessing Neuromotor Coordination in Schizophrenia Using Convolutional Neural Networks](https://mlsanity.com/paper/2110.04440/) (2021)
- [A Multimodal Framework for the Assessment of the Schizophrenia Spectrum](https://mlsanity.com/paper/2406.09706/) (2024)
- [Multimodal Depression Classification Using Articulatory Coordination Features And Hierarchical Attention Based Text Embeddings](https://mlsanity.com/paper/2202.06238/) (2022)
- [A multi-modal approach for identifying schizophrenia using cross-modal attention](https://mlsanity.com/paper/2309.15136/) (2023)
- [Speech-Based Estimation of Schizophrenia Severity Using Feature Fusion](https://mlsanity.com/paper/2411.06033/) (2024)
- [Self-supervised Multimodal Speech Representations for the Assessment of Schizophrenia Symptoms](https://mlsanity.com/paper/2409.09733/) (2024)

## Built on by (most cited first, 3 of 3)

- [Quantifying Articulatory Coordination as a Biomarker for Schizophrenia](https://mlsanity.com/paper/2511.03084/) (2025)
- [Articulation-Informed ASR: Integrating Articulatory Features into ASR via Auxiliary Speech Inversion and Cross-Attention Fusion](https://mlsanity.com/paper/2510.08585/) (2025)
- [Exploration of Perceptual Speech Features for Clinical Decision-Support in Mental Health Care](https://mlsanity.com/paper/2605.24678/) (2026)

## Abstract (arXiv)

Studies on schizophrenia assessments using deep learning typically treat it as a classification task to detect the presence or absence of the disorder, oversimplifying the condition and reducing its clinical applicability. This traditional approach overlooks the complexity of schizophrenia, limiting its practical value in healthcare settings. This study shifts the focus to individual symptom severity estimation using a multimodal approach that integrates speech, video, and text inputs. We develop unimodal models for each modality and a multimodal framework to improve accuracy and robustness. By capturing a more detailed symptom profile, this approach can help in enhancing diagnostic precision and support personalized treatment, offering a scalable and objective tool for mental health assessment.