# Facial Expression Recognition in the Deep Learning Era: A Systematic Multi-Criteria Review of Methods, Models, Datasets, Performance, Challenges, and Future Research Directions

arXiv: https://arxiv.org/abs/2606.08612 · HTML: https://mlsanity.com/paper/2606.08612/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 321 references that are in the map.
- First posted on arXiv 7 Jun 2026.

## Builds on (most cited first, 17 of 17)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [An Analysis of Deep Neural Network Models for Practical Applications](https://mlsanity.com/paper/1605.07678/) (2016)
- [Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition](https://mlsanity.com/paper/1811.07770/) (2018)
- [EMO-LLaMA: Enhancing Facial Emotion Understanding with Instruction Tuning](https://mlsanity.com/paper/2408.11424/) (2024)
- [NR-DFERNet: Noise-Robust Network for Dynamic Facial Expression Recognition](https://mlsanity.com/paper/2206.04975/) (2022)
- [PFLD: A Practical Facial Landmark Detector](https://mlsanity.com/paper/1902.10859/) (2019)
- [A Survey on Facial Expression Recognition of Static and Dynamic Emotions](https://mlsanity.com/paper/2408.15777/) (2024)
- [EmotioNet Challenge: Recognition of facial expressions of emotion in the wild](https://mlsanity.com/paper/1703.01210/) (2017)
- [Frequency-Aware Density Control via Reparameterization for High-Quality Rendering of 3D Gaussian Splatting](https://mlsanity.com/paper/2503.07000/) (2025)
- [FER-YOLO-Mamba: Facial Expression Detection and Classification Based on Selective State Space](https://mlsanity.com/paper/2405.01828/) (2024)
- [Computer Vision Estimation of Emotion Reaction Intensity in the Wild](https://mlsanity.com/paper/2303.10741/) (2023)
- [Detecting Localized Deepfake Manipulations Using Action Unit-Guided Video Representations](https://mlsanity.com/paper/2503.22121/) (2025)
- [Graph-Attention Network with Adversarial Domain Alignment for Robust Cross-Domain Facial Expression Recognition](https://mlsanity.com/paper/2512.00641/) (2025)
- [Human Reaction Intensity Estimation with Ensemble of Multi-task Networks](https://mlsanity.com/paper/2303.09240/) (2023)
- [EmoDiffusion: Enhancing Emotional 3D Facial Animation with Latent Diffusion Models](https://mlsanity.com/paper/2503.11028/) (2025)

## Abstract (arXiv)

Facial Expression Recognition (FER) has advanced rapidly over the last decade, driven by the shift from handcrafted descriptors and shallow classifiers to deep convolutional, attention-based, vision-language, and foundation-model architectures, and by the parallel growth of large-scale in-the-wild benchmarks spanning categorical, dimensional, compound, micro-expression, Action Unit (AU), and intensity-estimation tasks. Yet the deep learning-based FER landscape has so far been reviewed only along narrow task-, architecture-, or application-specific axes, leaving a holistic, systematically organized account of its recent advances missing. This survey addresses that gap with a comprehensive review of recent deep learning-based FER, explicitly linked to the wider Facial Affect Recognition (FAR) domain. Its main contributions are: a) A description of FER's evolution into five distinct phases, from handcrafted features and classical machine learning to attention-based, vision-language, and foundation-model approaches, with the key milestone works of each, b) A multi-criteria taxonomy analyzing the literature along seven complementary axes: recognition task, input modality, face pre-processing pipeline, network architecture, learning strategy, acquisition setting, and application domain, c) A per-criterion comparative analysis, with critical insights into the strengths and limitations of each category under in-the-wild conditions, d) A task-organized review of public FER datasets, with their annotation schemes, modalities, and evaluation protocols, e) A compilation of performance metrics and a per-task quantitative comparison of representative state-of-the-art methods on widely adopted benchmarks, and f) A discussion of current challenges and promising future directions.