# Deep Learning for Micro-expression Recognition: A Survey

arXiv: https://arxiv.org/abs/2107.02823 · HTML: https://mlsanity.com/paper/2107.02823/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 67 of its 237 references that are in the map.
- First posted on arXiv 6 Jul 2021.

## Builds on (most cited first, 30 of 67)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Deep Learning for Generic Object Detection: A Survey](https://mlsanity.com/paper/1809.02165/) (2018)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [WIDER FACE: A Face Detection Benchmark](https://mlsanity.com/paper/1511.06523/) (2015)
- [Large-Margin Softmax Loss for Convolutional Neural Networks](https://mlsanity.com/paper/1612.02295/) (2016)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Wider or Deeper: Revisiting the ResNet Model for Visual Recognition](https://mlsanity.com/paper/1611.10080/) (2016)
- [Context-aware Synthesis for Video Frame Interpolation](https://mlsanity.com/paper/1803.10967/) (2018)
- [Overcoming Classifier Imbalance for Long-tail Object Detection with Balanced Group Softmax](https://mlsanity.com/paper/2006.10408/) (2020)

## Built on by (most cited first, 30 of 43)

- [Benchmarking Micro-action Recognition: Dataset, Methods, and Applications](https://mlsanity.com/paper/2403.05234/) (2024)
- [GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognition](https://mlsanity.com/paper/2312.04293/) (2023)
- [Feature Representation Learning with Adaptive Displacement Generation and Transformer Fusion for Micro-Expression Recognition](https://mlsanity.com/paper/2304.04420/) (2023)
- [DFME: A New Benchmark for Dynamic Facial Micro-expression Recognition](https://mlsanity.com/paper/2301.00985/) (2023)
- [Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion](https://mlsanity.com/paper/2307.13463/) (2023)
- [Uncertain Facial Expression Recognition via Multi-task Assisted Correction](https://mlsanity.com/paper/2212.07144/) (2022)
- [Data Leakage and Evaluation Issues in Micro-Expression Analysis](https://mlsanity.com/paper/2211.11425/) (2022)
- [Learning to Rank Onset-Occurring-Offset Representations for Micro-Expression Recognition](https://mlsanity.com/paper/2310.04664/) (2023)
- [Geometric Graph Representation with Learnable Graph Structure and Adaptive AU Constraint for Micro-Expression Recognition](https://mlsanity.com/paper/2205.00380/) (2022)
- [Synergistic Spotting and Recognition of Micro-Expression via Temporal State Transition](https://mlsanity.com/paper/2409.09707/) (2024)
- [Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Survey](https://mlsanity.com/paper/2412.17616/) (2024)
- [Exploring Spatial-Temporal Dynamics in Event-based Facial Micro-Expression Analysis](https://mlsanity.com/paper/2508.11988/) (2025)
- [PESTalk: Speech-Driven 3D Facial Animation with Personalized Emotional Styles](https://mlsanity.com/paper/2512.05121/) (2025)
- [Rethinking the Learning Paradigm for Facial Expression Recognition](https://mlsanity.com/paper/2209.15402/) (2022)
- [Infused Suppression Of Magnification Artefacts For Micro-AU Detection](https://mlsanity.com/paper/2504.09326/) (2025)
- [MPFNet: A Multi-Prior Fusion Network with a Progressive Training Strategy for Micro-Expression Recognition](https://mlsanity.com/paper/2506.09735/) (2025)
- [Deep Insights of Learning based Micro Expression Recognition: A Perspective on Promises, Challenges and Research Needs](https://mlsanity.com/paper/2210.04935/) (2022)
- [Prior Information based Decomposition and Reconstruction Learning for Micro-Expression Recognition](https://mlsanity.com/paper/2303.01776/) (2023)
- [Rethinking Key-frame-based Micro-expression Recognition: A Robust and Accurate Framework Against Key-frame Errors](https://mlsanity.com/paper/2508.06640/) (2025)
- [Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action](https://mlsanity.com/paper/2606.15647/) (2026)
- [AMMSM: Adaptive Motion Magnification and Sparse Mamba for Micro-Expression Recognition](https://mlsanity.com/paper/2503.24057/) (2025)
- [ME-TST+: Micro-expression Analysis via Temporal State Transition with ROI Relationship Awareness](https://mlsanity.com/paper/2508.08082/) (2025)
- [Boosting Micro-Expression Analysis via Prior-Guided Video-Level Regression](https://mlsanity.com/paper/2508.18834/) (2025)
- [Dual Stream Independence Decoupling for True Emotion Recognition under Masked Expressions](https://mlsanity.com/paper/2603.16760/) (2026)
- [Exploring Interactions and Regulations in Collaborative Learning: An Interdisciplinary Multimodal Dataset](https://mlsanity.com/paper/2210.05419/) (2022)
- [Boosting Binary Neural Networks via Dynamic Thresholds Learning](https://mlsanity.com/paper/2211.02292/) (2022)
- [From Macro to Micro: Boosting micro-expression recognition via pre-training on macro-expression videos](https://mlsanity.com/paper/2405.16451/) (2024)
- [Three-Stream Temporal-Shift Attention Network Based on Self-Knowledge Distillation for Micro-Expression Recognition](https://mlsanity.com/paper/2406.17538/) (2024)
- [Enhancing Lie Detection Accuracy: A Comparative Study of Classic ML, CNN, and GCN Models using Audio-Visual Features](https://mlsanity.com/paper/2411.08885/) (2024)
- [Teleology-Driven Affective Computing: A Causal Framework for Sustained Well-Being](https://mlsanity.com/paper/2502.17172/) (2025)

## Abstract (arXiv)

Micro-expressions (MEs) are involuntary facial movements revealing people's hidden feelings in high-stake situations and have practical importance in medical treatment, national security, interrogations and many human-computer interaction systems. Early methods for MER mainly based on traditional appearance and geometry features. Recently, with the success of deep learning (DL) in various fields, neural networks have received increasing interests in MER. Different from macro-expressions, MEs are spontaneous, subtle, and rapid facial movements, leading to difficult data collection, thus have small-scale datasets. DL based MER becomes challenging due to above ME characters. To date, various DL approaches have been proposed to solve the ME issues and improve MER performance. In this survey, we provide a comprehensive review of deep micro-expression recognition (MER), including datasets, deep MER pipeline, and the bench-marking of most influential methods. This survey defines a new taxonomy for the field, encompassing all aspects of MER based on DL. For each aspect, the basic approaches and advanced developments are summarized and discussed. In addition, we conclude the remaining challenges and and potential directions for the design of robust deep MER systems. To the best of our knowledge, this is the first survey of deep MER methods, and this survey can serve as a reference point for future MER research.