# GAME: Generalized deep learning model towards multimodal data integration for early screening of adolescent mental disorders

arXiv: https://arxiv.org/abs/2309.10077 · HTML: https://mlsanity.com/paper/2309.10077/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 95 references that are in the map.
- First posted on arXiv 18 Sep 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [What Makes Multi-modal Learning Better than Single (Provably)](https://mlsanity.com/paper/2106.04538/) (2021)
- [Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training](https://mlsanity.com/paper/2104.01027/) (2021)
- [Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs](https://mlsanity.com/paper/1907.06724/) (2019)
- [Distributed and parallel time series feature extraction for industrial big data applications](https://mlsanity.com/paper/1610.07717/) (2016)
- [EmbraceNet: A robust deep learning architecture for multimodal classification](https://mlsanity.com/paper/1904.09078/) (2019)
- [PERT: Pre-training BERT with Permuted Language Model](https://mlsanity.com/paper/2203.06906/) (2022)
- [RCMNet: A deep learning model assists CAR-T therapy for leukemia](https://mlsanity.com/paper/2205.04230/) (2022)
- [Mixed-UNet: Refined Class Activation Mapping for Weakly-Supervised Semantic Segmentation with Multi-scale Inference](https://mlsanity.com/paper/2205.04227/) (2022)
- [A Machine Learning Analysis of COVID-19 Mental Health Data](https://mlsanity.com/paper/2112.00227/) (2021)
- [Artificial Intelligence and the Future of Psychiatry: Insights from a Global Physician Survey](https://mlsanity.com/paper/1907.12386/) (2019)
- [Real-time Pupil Tracking from Monocular Video for Digital Puppetry](https://mlsanity.com/paper/2006.11341/) (2020)

## Built on by (most cited first, 1 of 1)

- [MAJORScore: A Novel Metric for Evaluating Multimodal Relevance via Joint Representation](https://mlsanity.com/paper/2509.21365/) (2025)

## Abstract (arXiv)

The timely identification of mental disorders in adolescents is a global public health challenge.Single factor is difficult to detect the abnormality due to its complex and subtle nature. Additionally, the generalized multimodal Computer-Aided Screening (CAS) systems with interactive robots for adolescent mental disorders are not available. Here, we design an android application with mini-games and chat recording deployed in a portable robot to screen 3,783 middle school students and construct the multimodal screening dataset, including facial images, physiological signs, voice recordings, and textual transcripts.We develop a model called GAME (Generalized Model with Attention and Multimodal EmbraceNet) with novel attention mechanism that integrates cross-modal features into the model. GAME evaluates adolescent mental conditions with high accuracy (73.34%-92.77%) and F1-Score (71.32%-91.06%).We find each modality contributes dynamically to the mental disorders screening and comorbidities among various mental disorders, indicating the feasibility of explainable model. This study provides a system capable of acquiring multimodal information and constructs a generalized multimodal integration algorithm with novel attention mechanisms for the early screening of adolescent mental disorders.