# Feasibility of Detecting Cognitive Impairment and Psychological Well-being among Older Adults Using Facial, Acoustic, Linguistic, and Cardiovascular Patterns Derived from Remote Conversations

arXiv: https://arxiv.org/abs/2412.14194 · HTML: https://mlsanity.com/paper/2412.14194/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 120 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Builds on (most cited first, 22 of 22)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis](https://mlsanity.com/paper/1904.01906/) (2019)
- [Fairlearn: Assessing and Improving Fairness of AI Systems](https://mlsanity.com/paper/2303.16626/) (2023)
- [DeAR: Debiasing Vision-Language Models with Additive Residuals](https://mlsanity.com/paper/2303.10431/) (2023)
- [Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment](https://mlsanity.com/paper/1803.05588/) (2018)
- [MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks](https://mlsanity.com/paper/2212.08158/) (2022)
- [Face2PPG: An unsupervised pipeline for blood volume pulse extraction from faces](https://mlsanity.com/paper/2202.04101/) (2022)
- [MC-ViViT: Multi-branch Classifier-ViViT to detect Mild Cognitive Impairment in older adults using facial videos](https://mlsanity.com/paper/2304.05292/) (2023)
- [A Comparison of Personalized and Generalized Approaches to Emotion Recognition Using Consumer Wearable Devices: Machine Learning Study](https://mlsanity.com/paper/2308.14245/) (2023)
- [A Survey of Few-Shot Learning for Biomedical Time Series](https://mlsanity.com/paper/2405.02485/) (2024)
- [Longitudinal Speech Biomarkers for Automated Alzheimer's Detection](https://mlsanity.com/paper/2111.11859/) (2021)
- [Refining Automatic Speech Recognition System for older adults](https://mlsanity.com/paper/2011.08346/) (2020)
- [Feasibility of assessing cognitive impairment via distributed camera network and privacy-preserving edge computing](https://mlsanity.com/paper/2408.10442/) (2024)

## Built on by (most cited first, 2 of 2)

- [Passive Dementia Screening via Facial Temporal Micro-Dynamics Analysis of In-the-Wild Talking-Head Video](https://mlsanity.com/paper/2511.13802/) (2025)
- [TAMI: Temporally Aligned, Missingness-Aware, and Interpretable Multimodal Fusion for Mental Health Assessment in Older Adults with Mild Cognitive Impairment](https://mlsanity.com/paper/2608.30857/) (2026)

## Abstract (arXiv)

The aging society urgently requires scalable methods to monitor cognitive decline and identify social and psychological factors indicative of dementia risk in older adults. Our machine learning (ML) models captured facial, acoustic, linguistic, and cardiovascular features from 39 older adults with normal cognition or Mild Cognitive Impairment (MCI), derived from remote video conversations and quantified their cognitive status, social isolation, neuroticism, and psychological well-being. Our model could distinguish Clinical Dementia Rating Scale (CDR) of 0.5 (vs. 0) with 0.77 area under the receiver operating characteristic curve (AUC), social isolation with 0.74 AUC, social satisfaction with 0.75 AUC, psychological well-being with 0.72 AUC, and negative affect with 0.74 AUC. Our feature importance analysis showed that speech and language patterns were useful for quantifying cognitive impairment, whereas facial expressions and cardiovascular patterns were useful for quantifying social and psychological well-being. Our bias analysis showed that the best-performing models for quantifying psychological well-being and cognitive states in older adults exhibited significant biases concerning their age, sex, disease condition, and education levels. Our comprehensive analysis shows the feasibility of monitoring the cognitive and psychological health of older adults, as well as the need for collecting largescale interview datasets of older adults to benefit from the latest advances in deep learning technologies to develop generalizable models across older adults with diverse demographic backgrounds and disease conditions.