# MC-ViViT: Multi-branch Classifier-ViViT to detect Mild Cognitive Impairment in older adults using facial videos

arXiv: https://arxiv.org/abs/2304.05292 · HTML: https://mlsanity.com/paper/2304.05292/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 93 references that are in the map.
- First posted on arXiv 11 Apr 2023.

## Builds on (most cited first, 17 of 17)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Multiview Transformers for Video Recognition](https://mlsanity.com/paper/2201.04288/) (2022)
- [Space-time Mixing Attention for Video Transformer](https://mlsanity.com/paper/2106.05968/) (2021)
- [Multimodal Engagement Analysis from Facial Videos in the Classroom](https://mlsanity.com/paper/2101.04215/) (2021)
- [Video-based Facial Expression Recognition using Graph Convolutional Networks](https://mlsanity.com/paper/2010.13386/) (2020)
- [Towards Training Stronger Video Vision Transformers for EPIC-KITCHENS-100 Action Recognition](https://mlsanity.com/paper/2106.05058/) (2021)
- [Mutual Information Regularized Identity-aware Facial ExpressionRecognition in Compressed Video](https://mlsanity.com/paper/2010.10637/) (2020)
- [Facial Expression Recognition with Swin Transformer](https://mlsanity.com/paper/2203.13472/) (2022)
- [Video Vision Transformers for Violence Detection](https://mlsanity.com/paper/2209.03561/) (2022)
- [Anomaly Detection using Edge Computing in Video Surveillance System: Review](https://mlsanity.com/paper/2107.02778/) (2021)
- [XnODR and XnIDR: Two Accurate and Fast Fully Connected Layers For Convolutional Neural Networks](https://mlsanity.com/paper/2111.10854/) (2021)
- [Handling Inter-class and Intra-class Imbalance in Class-imbalanced Learning](https://mlsanity.com/paper/2111.12791/) (2021)
- [An Efficient Multitask Neural Network for Face Alignment, Head Pose Estimation and Face Tracking](https://mlsanity.com/paper/2103.07615/) (2021)
- [TimeConvNets: A Deep Time Windowed Convolution Neural Network Design for Real-time Video Facial Expression Recognition](https://mlsanity.com/paper/2003.01791/) (2020)

## Built on by (most cited first, 11 of 11)

- [Detection of Mild Cognitive Impairment Using Facial Features in Video Conversations](https://mlsanity.com/paper/2308.15624/) (2023)
- [A Review of Deep Learning Approaches for Non-Invasive Cognitive Impairment Detection](https://mlsanity.com/paper/2410.19898/) (2024)
- [Linguistic-Based Mild Cognitive Impairment Detection Using Informative Loss](https://mlsanity.com/paper/2402.01690/) (2024)
- [Data Quality Matters: Suicide Intention Detection on Social Media Posts Using RoBERTa-CNN](https://mlsanity.com/paper/2402.02262/) (2024)
- [Feasibility of Detecting Cognitive Impairment and Psychological Well-being among Older Adults Using Facial, Acoustic, Linguistic, and Cardiovascular Patterns Derived from Remote Conversations](https://mlsanity.com/paper/2412.14194/) (2024)
- [Deep Insights into Cognitive Decline: A Survey of Leveraging Non-Intrusive Modalities with Deep Learning Techniques](https://mlsanity.com/paper/2410.18972/) (2024)
- [Fine-Tuning Video Transformers for Word-Level Bangla Sign Language: A Comparative Analysis for Classification Tasks](https://mlsanity.com/paper/2506.04367/) (2025)
- [Cross-Enhanced Multimodal Fusion of Eye-Tracking and Facial Features for Alzheimer's Disease Diagnosis](https://mlsanity.com/paper/2510.24777/) (2025)
- [Contrastive learning-based video quality assessment-jointed video vision transformer for video recognition](https://mlsanity.com/paper/2603.10965/) (2026)
- [TAMI: Temporally Aligned, Missingness-Aware, and Interpretable Multimodal Fusion for Mental Health Assessment in Older Adults with Mild Cognitive Impairment](https://mlsanity.com/paper/2608.30857/) (2026)
- [MT-ProtBERT: Multi-task Learning ProtBERT for Intrinsically Disordered Proteins Classification with Scarce Data](https://mlsanity.com/paper/2609.25334/) (2026)

## Abstract (arXiv)

Deep machine learning models including Convolutional Neural Networks (CNN) have been successful in the detection of Mild Cognitive Impairment (MCI) using medical images, questionnaires, and videos. This paper proposes a novel Multi-branch Classifier-Video Vision Transformer (MC-ViViT) model to distinguish MCI from those with normal cognition by analyzing facial features. The data comes from the I-CONECT, a behavioral intervention trial aimed at improving cognitive function by providing frequent video chats. MC-ViViT extracts spatiotemporal features of videos in one branch and augments representations by the MC module. The I-CONECT dataset is challenging as the dataset is imbalanced containing Hard-Easy and Positive-Negative samples, which impedes the performance of MC-ViViT. We propose a loss function for Hard-Easy and Positive-Negative Samples (HP Loss) by combining Focal loss and AD-CORRE loss to address the imbalanced problem. Our experimental results on the I-CONECT dataset show the great potential of MC-ViViT in predicting MCI with a high accuracy of 90.63% accuracy on some of the interview videos.