# Deep Facial Expression Recognition: A Survey

arXiv: https://arxiv.org/abs/1804.08348 · HTML: https://mlsanity.com/paper/1804.08348/

## Key facts

- Cited by 270 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2018.

## Built on by (most cited first, 30 of 270)

- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in Videos](https://mlsanity.com/paper/2203.09463/) (2022)
- [MAFW: A Large-scale, Multi-modal, Compound Affective Database for Dynamic Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2208.00847/) (2022)
- [MARLIN: Masked Autoencoder for facial video Representation LearnINg](https://mlsanity.com/paper/2211.06627/) (2022)
- [Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms](https://mlsanity.com/paper/2201.12728/) (2022)
- [Understanding and Mitigating Annotation Bias in Facial Expression Recognition](https://mlsanity.com/paper/2108.08504/) (2021)
- [Facial Expression Recognition with Visual Transformers and Attentional Selective Fusion](https://mlsanity.com/paper/2103.16854/) (2021)
- [Mitigating Dataset Harms Requires Stewardship: Lessons from 1000 Papers](https://mlsanity.com/paper/2108.02922/) (2021)
- [POSTER: A Pyramid Cross-Fusion Transformer Network for Facial Expression Recognition](https://mlsanity.com/paper/2204.04083/) (2022)
- [Affective Image Content Analysis: Two Decades Review and New Perspectives](https://mlsanity.com/paper/2106.16125/) (2021)
- [Prompting Visual-Language Models for Dynamic Facial Expression Recognition](https://mlsanity.com/paper/2308.13382/) (2023)
- [Deep Learning for Micro-expression Recognition: A Survey](https://mlsanity.com/paper/2107.02823/) (2021)
- [MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition](https://mlsanity.com/paper/2307.02227/) (2023)
- [Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2208.10335/) (2022)
- [The EMPATHIC Framework for Task Learning from Implicit Human Feedback](https://mlsanity.com/paper/2009.13649/) (2020)
- [GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognition](https://mlsanity.com/paper/2312.04293/) (2023)
- [Expression Snippet Transformer for Robust Video-based Facial Expression Recognition](https://mlsanity.com/paper/2109.08409/) (2021)
- [Learning Emotion Representations from Verbal and Nonverbal Communication](https://mlsanity.com/paper/2305.13500/) (2023)
- [FineCLIPER: Multi-modal Fine-grained CLIP for Dynamic Facial Expression Recognition with AdaptERs](https://mlsanity.com/paper/2407.02157/) (2024)
- [Spatio-Temporal Transformer for Dynamic Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2205.04749/) (2022)
- [CLIPER: A Unified Vision-Language Framework for In-the-Wild Facial Expression Recognition](https://mlsanity.com/paper/2303.00193/) (2023)
- [MVT: Mask Vision Transformer for Facial Expression Recognition in the wild](https://mlsanity.com/paper/2106.04520/) (2021)
- [NR-DFERNet: Noise-Robust Network for Dynamic Facial Expression Recognition](https://mlsanity.com/paper/2206.04975/) (2022)
- [Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Models](https://mlsanity.com/paper/2304.00600/) (2023)
- [Stimuli-Aware Visual Emotion Analysis](https://mlsanity.com/paper/2109.01812/) (2021)
- [Facial Affective Behavior Analysis with Instruction Tuning](https://mlsanity.com/paper/2404.05052/) (2024)
- [Robust Emotion Recognition in Context Debiasing](https://mlsanity.com/paper/2403.05963/) (2024)
- [The Multimodal Sentiment Analysis in Car Reviews (MuSe-CaR) Dataset: Collection, Insights and Improvements](https://mlsanity.com/paper/2101.06053/) (2021)
- [Levels of explainable artificial intelligence for human-aligned conversational explanations](https://mlsanity.com/paper/2107.03178/) (2021)
- [Global-Local Attention for Emotion Recognition](https://mlsanity.com/paper/2111.04129/) (2021)

## Abstract (arXiv)

With the transition of facial expression recognition (FER) from laboratory-controlled to challenging in-the-wild conditions and the recent success of deep learning techniques in various fields, deep neural networks have increasingly been leveraged to learn discriminative representations for automatic FER. Recent deep FER systems generally focus on two important issues: overfitting caused by a lack of sufficient training data and expression-unrelated variations, such as illumination, head pose and identity bias. In this paper, we provide a comprehensive survey on deep FER, including datasets and algorithms that provide insights into these intrinsic problems. First, we describe the standard pipeline of a deep FER system with the related background knowledge and suggestions of applicable implementations for each stage. We then introduce the available datasets that are widely used in the literature and provide accepted data selection and evaluation principles for these datasets. For the state of the art in deep FER, we review existing novel deep neural networks and related training strategies that are designed for FER based on both static images and dynamic image sequences, and discuss their advantages and limitations. Competitive performances on widely used benchmarks are also summarized in this section. We then extend our survey to additional related issues and application scenarios. Finally, we review the remaining challenges and corresponding opportunities in this field as well as future directions for the design of robust deep FER systems.