# An Exploration of Active Learning for Affective Digital Phenotyping

arXiv: https://arxiv.org/abs/2204.01915 · HTML: https://mlsanity.com/paper/2204.01915/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 76 references that are in the map.
- First posted on arXiv 5 Apr 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Activity Recognition with Moving Cameras and Few Training Examples: Applications for Detection of Autism-Related Headbanging](https://mlsanity.com/paper/2101.03478/) (2021)
- [Superpower Glass: Delivering Unobtrusive Real-time Social Cues in Wearable Systems](https://mlsanity.com/paper/2002.06581/) (2020)
- [Improved Digital Therapy for Developmental Pediatrics Using Domain-Specific Artificial Intelligence: Machine Learning Study](https://mlsanity.com/paper/2012.08678/) (2020)
- [Classification of Abnormal Hand Movement for Aiding in Autism Detection: Machine Learning Study](https://mlsanity.com/paper/2108.07917/) (2021)
- [Training Affective Computer Vision Models by Crowdsourcing Soft-Target Labels](https://mlsanity.com/paper/2101.03477/) (2021)
- [Training and Profiling a Pediatric Emotion Recognition Classifier on Mobile Devices](https://mlsanity.com/paper/2108.11754/) (2021)
- [Designing a Holistic At-Home Learning Aid for Autism](https://mlsanity.com/paper/2002.04263/) (2020)
- [Challenges and Opportunities for Machine Learning Classification of Behavior and Mental State from Images](https://mlsanity.com/paper/2201.11197/) (2022)
- [Classifying Autism from Crowdsourced Semi-Structured Speech Recordings: A Machine Learning Approach](https://mlsanity.com/paper/2201.00927/) (2022)

## Abstract (arXiv)

Some of the most severe bottlenecks preventing widespread development of machine learning models for human behavior include a dearth of labeled training data and difficulty of acquiring high quality labels. Active learning is a paradigm for using algorithms to computationally select a useful subset of data points to label using metrics for model uncertainty and data similarity. We explore active learning for naturalistic computer vision emotion data, a particularly heterogeneous and complex data space due to inherently subjective labels. Using frames collected from gameplay acquired from a therapeutic smartphone game for children with autism, we run a simulation of active learning using gameplay prompts as metadata to aid in the active learning process. We find that active learning using information generated during gameplay slightly outperforms random selection of the same number of labeled frames. We next investigate a method to conduct active learning with subjective data, such as in affective computing, and where multiple crowdsourced labels can be acquired for each image. Using the Child Affective Facial Expression (CAFE) dataset, we simulate an active learning process for crowdsourcing many labels and find that prioritizing frames using the entropy of the crowdsourced label distribution results in lower categorical cross-entropy loss compared to random frame selection. Collectively, these results demonstrate pilot evaluations of two novel active learning approaches for subjective affective data collected in noisy settings.