# CNN-based Facial Affect Analysis on Mobile Devices

arXiv: https://arxiv.org/abs/1807.08775 · HTML: https://mlsanity.com/paper/1807.08775/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jul 2018.

## Built on by (most cited first, 5 of 5)

- [Face Trees for Expression Recognition](https://mlsanity.com/paper/2112.02487/) (2021)
- [CIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition](https://mlsanity.com/paper/2208.07221/) (2022)
- [FaceTopoNet: Facial Expression Recognition using Face Topology Learning](https://mlsanity.com/paper/2209.06322/) (2022)
- [Learning Diversified Feature Representations for Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2210.09381/) (2022)
- [A cross-modal network for facial expression recognition](https://mlsanity.com/paper/2605.04439/) (2026)

## Abstract (arXiv)

This paper focuses on the design, deployment and evaluation of Convolutional Neural Network (CNN) architectures for facial affect analysis on mobile devices. Unlike traditional CNN approaches, models deployed to mobile devices must minimise storage requirements while retaining high performance. We therefore propose three variants of established CNN architectures and comparatively evaluate them on a large, in-the-wild benchmark dataset of facial images. Our results show that the proposed architectures retain similar performance to the dataset baseline while minimising storage requirements: achieving 58% accuracy for eight-class emotion classification and average RMSE of 0.39 for valence/arousal prediction. To demonstrate the feasibility of deploying these models for real-world applications, we implement a music recommendation interface based on predicted user affect. Although the CNN models were not trained in the context of music recommendation, our case study shows that: (i) the trained models achieve similar prediction performance to the benchmark dataset, and (ii) users tend to positively rate the song recommendations provided by the interface. Average runtime of the deployed models on an iPhone 6S equates to ~45 fps, suggesting that the proposed architectures are also well suited for real-time deployment on video streams.