# Facial Expression Recognition using Convolutional Neural Networks: State of the Art

arXiv: https://arxiv.org/abs/1612.02903 · HTML: https://mlsanity.com/paper/1612.02903/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Dec 2016.

## Built on by (most cited first, 17 of 17)

- [Facial Emotion Recognition: State of the Art Performance on FER2013](https://mlsanity.com/paper/2105.03588/) (2021)
- [Significance tests of feature relevance for a black-box learner](https://mlsanity.com/paper/2103.04985/) (2021)
- [Local Multi-Head Channel Self-Attention for Facial Expression Recognition](https://mlsanity.com/paper/2111.07224/) (2021)
- [DMCNet: Diversified Model Combination Network for Understanding Engagement from Video Screengrabs](https://mlsanity.com/paper/2204.06454/) (2022)
- [Training and Profiling a Pediatric Emotion Recognition Classifier on Mobile Devices](https://mlsanity.com/paper/2108.11754/) (2021)
- [CIAO! A Contrastive Adaptation Mechanism for Non-Universal Facial Expression Recognition](https://mlsanity.com/paper/2208.07221/) (2022)
- [Regional Attention Network (RAN) for Head Pose and Fine-grained Gesture Recognition](https://mlsanity.com/paper/2101.06634/) (2021)
- [NL-CNN: A Resources-Constrained Deep Learning Model based on Nonlinear Convolution](https://mlsanity.com/paper/2102.00227/) (2021)
- [Recognizing Facial Expressions in the Wild using Multi-Architectural Representations based Ensemble Learning with Distillation](https://mlsanity.com/paper/2106.16126/) (2021)
- [Bayesian Convolutional Neural Networks for Seven Basic Facial Expression Classifications](https://mlsanity.com/paper/2107.04834/) (2021)
- [Facial Expressions Recognition with Convolutional Neural Networks](https://mlsanity.com/paper/2107.08640/) (2021)
- [Research on facial expression recognition based on Multimodal data fusion and neural network](https://mlsanity.com/paper/2109.12724/) (2021)
- [Interruptions detection in video conferences](https://mlsanity.com/paper/2303.02052/) (2023)
- [InFER: A Multi-Ethnic Indian Facial Expression Recognition Dataset](https://mlsanity.com/paper/2310.00287/) (2023)
- [Facial Emotion Recognition using CNN in PyTorch](https://mlsanity.com/paper/2312.10818/) (2023)
- [Leaving Some Facial Features Behind](https://mlsanity.com/paper/2411.00824/) (2024)
- [Facial Expression Recognition Using Residual Masking Network](https://mlsanity.com/paper/2603.05937/) (2026)

## Abstract (arXiv)

The ability to recognize facial expressions automatically enables novel applications in human-computer interaction and other areas. Consequently, there has been active research in this field, with several recent works utilizing Convolutional Neural Networks (CNNs) for feature extraction and inference. These works differ significantly in terms of CNN architectures and other factors. Based on the reported results alone, the performance impact of these factors is unclear. In this paper, we review the state of the art in image-based facial expression recognition using CNNs and highlight algorithmic differences and their performance impact. On this basis, we identify existing bottlenecks and consequently directions for advancing this research field. Furthermore, we demonstrate that overcoming one of these bottlenecks - the comparatively basic architectures of the CNNs utilized in this field - leads to a substantial performance increase. By forming an ensemble of modern deep CNNs, we obtain a FER2013 test accuracy of 75.2%, outperforming previous works without requiring auxiliary training data or face registration.