# SchiNet: Automatic Estimation of Symptoms of Schizophrenia from Facial Behaviour Analysis

arXiv: https://arxiv.org/abs/1808.02531 · HTML: https://mlsanity.com/paper/1808.02531/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Aug 2018.

## Built on by (most cited first, 16 of 16)

- [Prompting Visual-Language Models for Dynamic Facial Expression Recognition](https://mlsanity.com/paper/2308.13382/) (2023)
- [Attribute-preserving Face Dataset Anonymization via Latent Code Optimization](https://mlsanity.com/paper/2303.11296/) (2023)
- [Vision Transformer with Attentive Pooling for Robust Facial Expression Recognition](https://mlsanity.com/paper/2212.05463/) (2022)
- [EmoCLIP: A Vision-Language Method for Zero-Shot Video Facial Expression Recognition](https://mlsanity.com/paper/2310.16640/) (2023)
- [Enhancing Zero-Shot Facial Expression Recognition by LLM Knowledge Transfer](https://mlsanity.com/paper/2405.19100/) (2024)
- [BAH Dataset for Ambivalence/Hesitancy Recognition in Videos for Digital Behavioural Change](https://mlsanity.com/paper/2505.19328/) (2025)
- [ResEmoteNet: Bridging Accuracy and Loss Reduction in Facial Emotion Recognition](https://mlsanity.com/paper/2409.10545/) (2024)
- [Learning from Label Relationships in Human Affect](https://mlsanity.com/paper/2207.05577/) (2022)
- [AFFDEX 2.0: A Real-Time Facial Expression Analysis Toolkit](https://mlsanity.com/paper/2202.12059/) (2022)
- [Which CNNs and Training Settings to Choose for Action Unit Detection? A Study Based on a Large-Scale Dataset](https://mlsanity.com/paper/2111.08320/) (2021)
- [Choose Settings Carefully: Comparing Action Unit detection at Different Settings Using a Large-Scale Dataset](https://mlsanity.com/paper/2111.08324/) (2021)
- [Kernel function impact on convolutional neural networks](https://mlsanity.com/paper/2302.10266/) (2023)
- [Automatic Detection of Sentimentality from Facial Expressions](https://mlsanity.com/paper/2209.04908/) (2022)
- [PainNet: Statistical Relation Network with Episode-Based Training for Pain Estimation](https://mlsanity.com/paper/2504.06257/) (2025)
- [Controllable Localized Face Anonymization Via Diffusion Inpainting](https://mlsanity.com/paper/2509.14866/) (2025)
- [Text-guided Weakly Supervised Framework for Dynamic Facial Expression Recognition](https://mlsanity.com/paper/2511.10958/) (2025)

## Abstract (arXiv)

Patients with schizophrenia often display impairments in the expression of emotion and speech and those are observed in their facial behaviour. Automatic analysis of patients' facial expressions that is aimed at estimating symptoms of schizophrenia has received attention recently. However, the datasets that are typically used for training and evaluating the developed methods, contain only a small number of patients (4-34) and are recorded while the subjects were performing controlled tasks such as listening to life vignettes, or answering emotional questions. In this paper, we use videos of professional-patient interviews, in which symptoms were assessed in a standardised way as they should/may be assessed in practice, and which were recorded in realistic conditions (i.e. varying illumination levels and camera viewpoints) at the patients' homes or at mental health services. We automatically analyse the facial behaviour of 91 out-patients - this is almost 3 times the number of patients in other studies - and propose SchiNet, a novel neural network architecture that estimates expression-related symptoms in two different assessment interviews. We evaluate the proposed SchiNet for patient-independent prediction of symptoms of schizophrenia. Experimental results show that some automatically detected facial expressions are significantly correlated to symptoms of schizophrenia, and that the proposed network for estimating symptom severity delivers promising results.