# A Robust Framework for Deep Learning Approaches to Facial Emotion Recognition and Evaluation

arXiv: https://arxiv.org/abs/2201.12705 · HTML: https://mlsanity.com/paper/2201.12705/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 30 Jan 2022.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [Addressing Failure Prediction by Learning Model Confidence](https://mlsanity.com/paper/1910.04851/) (2019)
- [Facial Emotion Recognition: State of the Art Performance on FER2013](https://mlsanity.com/paper/2105.03588/) (2021)
- [Applications of Recurrent Neural Network for Biometric Authentication & Anomaly Detection](https://mlsanity.com/paper/2109.05701/) (2021)
- [Machine Learning Algorithms In User Authentication Schemes](https://mlsanity.com/paper/2110.07826/) (2021)
- [A Modern Analysis of Aging Machine Learning Based IoT Cybersecurity Methods](https://mlsanity.com/paper/2110.07832/) (2021)
- [Two-level Attention with Two-stage Multi-task Learning for Facial Emotion Recognition](https://mlsanity.com/paper/1811.12139/) (2018)
- [Continuous Authentication Using Mouse Movements, Machine Learning, and Minecraft](https://mlsanity.com/paper/2110.11080/) (2021)
- [The Benefits of Edge Computing in Healthcare, Smart Cities, and IoT](https://mlsanity.com/paper/2112.01250/) (2021)

## Built on by (most cited first, 3 of 3)

- [Evaluation of a User Authentication Schema Using Behavioral Biometrics and Machine Learning](https://mlsanity.com/paper/2205.08371/) (2022)
- [Multimodal Feature Extraction and Fusion for Emotional Reaction Intensity Estimation and Expression Classification in Videos with Transformers](https://mlsanity.com/paper/2303.09164/) (2023)
- [Mitigating Presentation Attack using DCGAN and Deep CNN](https://mlsanity.com/paper/2207.00161/) (2022)

## Abstract (arXiv)

Facial emotion recognition is a vast and complex problem space within the domain of computer vision and thus requires a universally accepted baseline method with which to evaluate proposed models. While test datasets have served this purpose in the academic sphere real world application and testing of such models lacks any real comparison. Therefore we propose a framework in which models developed for FER can be compared and contrasted against one another in a constant standardized fashion. A lightweight convolutional neural network is trained on the AffectNet dataset a large variable dataset for facial emotion recognition and a web application is developed and deployed with our proposed framework as a proof of concept. The CNN is embedded into our application and is capable of instant real time facial emotion recognition. When tested on the AffectNet test set this model achieves high accuracy for emotion classification of eight different emotions. Using our framework the validity of this model and others can be properly tested by evaluating a model efficacy not only based on its accuracy on a sample test dataset, but also on in the wild experiments. Additionally, our application is built with the ability to save and store any image captured or uploaded to it for emotion recognition, allowing for the curation of more quality and diverse facial emotion recognition datasets.