# Customizable Avatars with Dynamic Facial Action Coded Expressions (CADyFACE) for Improved User Engagement

arXiv: https://arxiv.org/abs/2403.07314 · HTML: https://mlsanity.com/paper/2403.07314/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 73 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 7 of 7)

- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [ABAW: Learning from Synthetic Data & Multi-Task Learning Challenges](https://mlsanity.com/paper/2207.01138/) (2022)
- [JhatAA-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attention](https://mlsanity.com/paper/2003.08834/) (2020)
- [AU R-CNN: Encoding Expert Prior Knowledge into R-CNN for Action Unit Detection](https://mlsanity.com/paper/1812.05788/) (2018)
- [Can Autism be Catered with Artificial Intelligence-Assisted Intervention Technology? A Literature Review](https://mlsanity.com/paper/1803.05181/) (2018)
- [On Graphical Models and Convex Geometry](https://mlsanity.com/paper/2106.14255/) (2021)
- [Deep Adaptation of Adult-Child Facial Expressions by Fusing Landmark Features](https://mlsanity.com/paper/2209.08614/) (2022)

## Abstract (arXiv)

Customizable 3D avatar-based facial expression stimuli may improve user engagement in behavioral biomarker discovery and therapeutic intervention for autism, Alzheimer's disease, facial palsy, and more. However, there is a lack of customizable avatar-based stimuli with Facial Action Coding System (FACS) action unit (AU) labels. Therefore, this study focuses on (1) FACS-labeled, customizable avatar-based expression stimuli for maintaining subjects' engagement, (2) learning-based measurements that quantify subjects' facial responses to such stimuli, and (3) validation of constructs represented by stimulus-measurement pairs. We propose Customizable Avatars with Dynamic Facial Action Coded Expressions (CADyFACE) labeled with AUs by a certified FACS expert. To measure subjects' AUs in response to CADyFACE, we propose a novel Beta-guided Correlation and Multi-task Expression learning neural network (BeCoME-Net) for multi-label AU detection. The beta-guided correlation loss encourages feature correlation with AUs while discouraging correlation with subject identities for improved generalization. We train BeCoME-Net for unilateral and bilateral AU detection and compare with state-of-the-art approaches. To assess construct validity of CADyFACE and BeCoME-Net, twenty healthy adult volunteers complete expression recognition and mimicry tasks in an online feasibility study while webcam-based eye-tracking and video are collected. We test validity of multiple constructs, including face preference during recognition and AUs during mimicry.