# Experimenting with Affective Computing Models in Video Interviews with Spanish-speaking Older Adults

arXiv: https://arxiv.org/abs/2501.16870 · HTML: https://mlsanity.com/paper/2501.16870/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 28 Jan 2025.

## Builds on (most cited first, 14 of 14)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [Towards Fast, Accurate and Stable 3D Dense Face Alignment](https://mlsanity.com/paper/2009.09960/) (2020)
- [MARLIN: Masked Autoencoder for facial video Representation LearnINg](https://mlsanity.com/paper/2211.06627/) (2022)
- [RoBERTuito: a pre-trained language model for social media text in Spanish](https://mlsanity.com/paper/2111.09453/) (2021)
- [pysentimiento: A Python Toolkit for Opinion Mining and Social NLP tasks](https://mlsanity.com/paper/2106.09462/) (2021)
- [Artificial Emotional Intelligence in Socially Assistive Robots for Older Adults: A Pilot Study](https://mlsanity.com/paper/2201.11167/) (2022)
- [Towards affective computing that works for everyone](https://mlsanity.com/paper/2309.10780/) (2023)
- [Exploring Emotion Expression Recognition in Older Adults Interacting with a Virtual Coach](https://mlsanity.com/paper/2311.05567/) (2023)
- [A Pilot Study on Using an Intelligent Life-like Robot as a Companion for Elderly Individuals with Dementia and Depression](https://mlsanity.com/paper/1712.02881/) (2017)
- [The EMPATHIC Project: Building an Expressive, Advanced Virtual Coach to Improve Independent Healthy-Life-Years of the Elderly](https://mlsanity.com/paper/2104.13836/) (2021)

## Abstract (arXiv)

Understanding emotional signals in older adults is crucial for designing virtual assistants that support their well-being. However, existing affective computing models often face significant limitations: (1) limited availability of datasets representing older adults, especially in non-English-speaking populations, and (2) poor generalization of models trained on younger or homogeneous demographics. To address these gaps, this study evaluates state-of-the-art affective computing models -- including facial expression recognition, text sentiment analysis, and smile detection -- using videos of older adults interacting with either a person or a virtual avatar. As part of this effort, we introduce a novel dataset featuring Spanish-speaking older adults engaged in human-to-human video interviews. Through three comprehensive analyses, we investigate (1) the alignment between human-annotated labels and automatic model outputs, (2) the relationships between model outputs across different modalities, and (3) individual variations in emotional signals. Using both the Wizard of Oz (WoZ) dataset and our newly collected dataset, we uncover limited agreement between human annotations and model predictions, weak consistency across modalities, and significant variability among individuals. These findings highlight the shortcomings of generalized emotion perception models and emphasize the need of incorporating personal variability and cultural nuances into future systems.