# Training and Profiling a Pediatric Emotion Recognition Classifier on Mobile Devices

arXiv: https://arxiv.org/abs/2108.11754 · HTML: https://mlsanity.com/paper/2108.11754/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 77 references that are in the map.
- First posted on arXiv 22 Aug 2021.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [From Facial Expression Recognition to Interpersonal Relation Prediction](https://mlsanity.com/paper/1609.06426/) (2016)
- [Facial Expression Recognition using Convolutional Neural Networks: State of the Art](https://mlsanity.com/paper/1612.02903/) (2016)
- [Superpower Glass: Delivering Unobtrusive Real-time Social Cues in Wearable Systems](https://mlsanity.com/paper/2002.06581/) (2020)
- [Improved Digital Therapy for Developmental Pediatrics Using Domain-Specific Artificial Intelligence: Machine Learning Study](https://mlsanity.com/paper/2012.08678/) (2020)
- [EmotionNet Nano: An Efficient Deep Convolutional Neural Network Design for Real-time Facial Expression Recognition](https://mlsanity.com/paper/2006.15759/) (2020)

## Built on by (most cited first, 5 of 5)

- [Challenges and Opportunities for Machine Learning Classification of Behavior and Mental State from Images](https://mlsanity.com/paper/2201.11197/) (2022)
- [Computer Vision Estimation of Emotion Reaction Intensity in the Wild](https://mlsanity.com/paper/2303.10741/) (2023)
- [Classifying Autism from Crowdsourced Semi-Structured Speech Recordings: A Machine Learning Approach](https://mlsanity.com/paper/2201.00927/) (2022)
- [An Exploration of Active Learning for Affective Digital Phenotyping](https://mlsanity.com/paper/2204.01915/) (2022)
- [Personalization of Stress Mobile Sensing using Self-Supervised Learning](https://mlsanity.com/paper/2308.02731/) (2023)

## Abstract (arXiv)

Implementing automated emotion recognition on mobile devices could provide an accessible diagnostic and therapeutic tool for those who struggle to recognize emotion, including children with developmental behavioral conditions such as autism. Although recent advances have been made in building more accurate emotion classifiers, existing models are too computationally expensive to be deployed on mobile devices. In this study, we optimized and profiled various machine learning models designed for inference on edge devices and were able to match previous state of the art results for emotion recognition on children. Our best model, a MobileNet-V2 network pre-trained on ImageNet, achieved 65.11% balanced accuracy and 64.19% F1-score on CAFE, while achieving a 45-millisecond inference latency on a Motorola Moto G6 phone. This balanced accuracy is only 1.79% less than the current state of the art for CAFE, which used a model that contains 26.62x more parameters and was unable to run on the Moto G6, even when fully optimized. This work validates that with specialized design and optimization techniques, machine learning models can become lightweight enough for deployment on mobile devices and still achieve high accuracies on difficult image classification tasks.