# Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms arXiv: https://arxiv.org/abs/2201.12728 ยท HTML: https://mlsanity.com/paper/2201.12728/ ## Key facts - Cited by 51 papers in the map (data through 14 Oct 2026). - Builds on 11 of its 126 references that are in the map. - First posted on arXiv 30 Jan 2022. ## Builds on (most cited first, 11 of 11) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018) - [Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges](https://mlsanity.com/paper/1803.07517/) (2018) - [Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-related Applications](https://mlsanity.com/paper/1606.03237/) (2016) - [Towards Reading Hidden Emotions: A comparative Study of Spontaneous Micro-expression Spotting and Recognition Methods](https://mlsanity.com/paper/1511.00423/) (2015) - [Enriched Long-term Recurrent Convolutional Network for Facial Micro-Expression Recognition](https://mlsanity.com/paper/1805.08417/) (2018) - [Spatiotemporal Recurrent Convolutional Networks for Recognizing Spontaneous Micro-expressions](https://mlsanity.com/paper/1901.04656/) (2019) - [LEARNet Dynamic Imaging Network for Micro Expression Recognition](https://mlsanity.com/paper/1904.09410/) (2019) - [A Survey of Automatic Facial Micro-expression Analysis: Databases, Methods and Challenges](https://mlsanity.com/paper/1806.05781/) (2018) - [Multi-Directional Multi-Level Dual-Cross Patterns for Robust Face Recognition](https://mlsanity.com/paper/1401.5311/) (2014) - [Facial Micro-Expression Spotting and Recognition using Time Contrasted Feature with Visual Memory](https://mlsanity.com/paper/1902.03514/) (2019) ## Built on by (most cited first, 30 of 51) - [FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in Videos](https://mlsanity.com/paper/2203.09463/) (2022) - [Micron-BERT: BERT-based Facial Micro-Expression Recognition](https://mlsanity.com/paper/2304.03195/) (2023) - [Feature Representation Learning with Adaptive Displacement Generation and Transformer Fusion for Micro-Expression Recognition](https://mlsanity.com/paper/2304.04420/) (2023) - [DFME: A New Benchmark for Dynamic Facial Micro-expression Recognition](https://mlsanity.com/paper/2301.00985/) (2023) - [Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion](https://mlsanity.com/paper/2307.13463/) (2023) - [Towards Online Multi-Modal Social Interaction Understanding](https://mlsanity.com/paper/2503.19851/) (2025) - [MOL: Joint Estimation of Micro-Expression, Optical Flow, and Landmark via Transformer-Graph-Style Convolution](https://mlsanity.com/paper/2506.14511/) (2025) - [Uncertain Facial Expression Recognition via Multi-task Assisted Correction](https://mlsanity.com/paper/2212.07144/) (2022) - [Neuromorphic Face Analysis: a Survey](https://mlsanity.com/paper/2402.11631/) (2024) - [Data Leakage and Evaluation Issues in Micro-Expression Analysis](https://mlsanity.com/paper/2211.11425/) (2022) - [Learning to Rank Onset-Occurring-Offset Representations for Micro-Expression Recognition](https://mlsanity.com/paper/2310.04664/) (2023) - [Geometric Graph Representation with Learnable Graph Structure and Adaptive AU Constraint for Micro-Expression Recognition](https://mlsanity.com/paper/2205.00380/) (2022) - [Synergistic Spotting and Recognition of Micro-Expression via Temporal State Transition](https://mlsanity.com/paper/2409.09707/) (2024) - [Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Survey](https://mlsanity.com/paper/2412.17616/) (2024) - [Weakly-supervised Micro- and Macro-expression Spotting Based on Multi-level Consistency](https://mlsanity.com/paper/2305.02734/) (2023) - [Infused Suppression Of Magnification Artefacts For Micro-AU Detection](https://mlsanity.com/paper/2504.09326/) (2025) - [Deep Insights of Learning based Micro Expression Recognition: A Perspective on Promises, Challenges and Research Needs](https://mlsanity.com/paper/2210.04935/) (2022) - [Weak Supervision with Arbitrary Single Frame for Micro- and Macro-expression Spotting](https://mlsanity.com/paper/2403.14240/) (2024) - [MEGC2025: Micro-Expression Grand Challenge on Spot Then Recognize and Visual Question Answering](https://mlsanity.com/paper/2506.15298/) (2025) - [FED-PsyAU: Privacy-Preserving Micro-Expression Recognition via Psychological AU Coordination and Dynamic Facial Motion Modeling](https://mlsanity.com/paper/2507.20557/) (2025) - [Rethinking Key-frame-based Micro-expression Recognition: A Robust and Accurate Framework Against Key-frame Errors](https://mlsanity.com/paper/2508.06640/) (2025) - [Evaluating and Correcting Human Annotation Bias in Dynamic Micro-Expression Recognition](https://mlsanity.com/paper/2603.04766/) (2026) - [Transferring Dual Stochastic Graph Convolutional Network for Facial Micro-expression Recognition](https://mlsanity.com/paper/2203.05208/) (2022) - [Detail-Enhanced Intra- and Inter-modal Interaction for Audio-Visual Emotion Recognition](https://mlsanity.com/paper/2405.16701/) (2024) - [AMMSM: Adaptive Motion Magnification and Sparse Mamba for Micro-Expression Recognition](https://mlsanity.com/paper/2503.24057/) (2025) - [Temporal and Spatial Feature Fusion Framework for Dynamic Micro Expression Recognition](https://mlsanity.com/paper/2505.16372/) (2025) - [ME-TST+: Micro-expression Analysis via Temporal State Transition with ROI Relationship Awareness](https://mlsanity.com/paper/2508.08082/) (2025) - [SocialFusion: Addressing Social Degradation in Pre-trained Vision-Language Models](https://mlsanity.com/paper/2512.01148/) (2025) - [Structured Prompting and LLM Ensembling for Multimodal Conversational Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2512.22603/) (2025) - [Micro-Expression Recognition Based on Attribute Information Embedding and Cross-modal Contrastive Learning](https://mlsanity.com/paper/2205.14643/) (2022) ## Abstract (arXiv) Unlike the conventional facial expressions, micro-expressions are involuntary and transient facial expressions capable of revealing the genuine emotions that people attempt to hide. Therefore, they can provide important information in a broad range of applications such as lie detection, criminal detection, etc. Since micro-expressions are transient and of low intensity, however, their detection and recognition is difficult and relies heavily on expert experiences. Due to its intrinsic particularity and complexity, video-based micro-expression analysis is attractive but challenging, and has recently become an active area of research. Although there have been numerous developments in this area, thus far there has been no comprehensive survey that provides researchers with a systematic overview of these developments with a unified evaluation. Accordingly, in this survey paper, we first highlight the key differences between macro- and micro-expressions, then use these differences to guide our research survey of video-based micro-expression analysis in a cascaded structure, encompassing the neuropsychological basis, datasets, features, spotting algorithms, recognition algorithms, applications and evaluation of state-of-the-art approaches. For each aspect, the basic techniques, advanced developments and major challenges are addressed and discussed. Furthermore, after considering the limitations of existing micro-expression datasets, we present and release a new dataset - called micro-and-macro expression warehouse (MMEW) - containing more video samples and more labeled emotion types. We then perform a unified comparison of representative methods on CAS(ME)2 for spotting, and on MMEW and SAMM for recognition, respectively. Finally, some potential future research directions are explored and outlined.