# Deep Insights of Learning based Micro Expression Recognition: A Perspective on Promises, Challenges and Research Needs

arXiv: https://arxiv.org/abs/2210.04935 · HTML: https://mlsanity.com/paper/2210.04935/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 103 references that are in the map.
- First posted on arXiv 10 Oct 2022.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation](https://mlsanity.com/paper/1901.02985/) (2019)
- [AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challenge](https://mlsanity.com/paper/1605.01600/) (2016)
- [M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues](https://mlsanity.com/paper/1911.05659/) (2019)
- [Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms](https://mlsanity.com/paper/2201.12728/) (2022)
- [EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle](https://mlsanity.com/paper/2003.06692/) (2020)
- [Learning-based Video Motion Magnification](https://mlsanity.com/paper/1804.02684/) (2018)
- [OFF-ApexNet on Micro-expression Recognition System](https://mlsanity.com/paper/1805.08699/) (2018)
- [Deep Learning for Micro-expression Recognition: A Survey](https://mlsanity.com/paper/2107.02823/) (2021)
- [Shallow Triple Stream Three-dimensional CNN (STSTNet) for Micro-expression Recognition](https://mlsanity.com/paper/1902.03634/) (2019)
- [Towards Reading Hidden Emotions: A comparative Study of Spontaneous Micro-expression Spotting and Recognition Methods](https://mlsanity.com/paper/1511.00423/) (2015)
- [Revealing the Invisible with Model and Data Shrinking for Composite-database Micro-expression Recognition](https://mlsanity.com/paper/2006.09674/) (2020)
- [Enriched Long-term Recurrent Convolutional Network for Facial Micro-Expression Recognition](https://mlsanity.com/paper/1805.08417/) (2018)
- [Spatiotemporal Recurrent Convolutional Networks for Recognizing Spontaneous Micro-expressions](https://mlsanity.com/paper/1901.04656/) (2019)
- [Feature refinement: An expression-specific feature learning and fusion method for micro-expression recognition](https://mlsanity.com/paper/2101.04838/) (2021)
- [LEARNet Dynamic Imaging Network for Micro Expression Recognition](https://mlsanity.com/paper/1904.09410/) (2019)
- [MER-GCN: Micro Expression Recognition Based on Relation Modeling with Graph Convolutional Network](https://mlsanity.com/paper/2004.08915/) (2020)
- [Spontaneous Facial Micro-Expression Recognition using 3D Spatiotemporal Convolutional Neural Networks](https://mlsanity.com/paper/1904.01390/) (2019)
- [A Survey of Automatic Facial Micro-expression Analysis: Databases, Methods and Challenges](https://mlsanity.com/paper/1806.05781/) (2018)
- [An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge](https://mlsanity.com/paper/2012.11307/) (2020)
- [Micro-Attention for Micro-Expression recognition](https://mlsanity.com/paper/1811.02360/) (2018)
- [Micro-expression spotting: A new benchmark](https://mlsanity.com/paper/2007.12421/) (2020)
- [A Review on Facial Micro-Expressions Analysis: Datasets, Features and Metrics](https://mlsanity.com/paper/1805.02397/) (2018)
- [AffectiveNet: Affective-Motion Feature Learningfor Micro Expression Recognition](https://mlsanity.com/paper/2104.07569/) (2021)

## Built on by (most cited first, 2 of 2)

- [Data Leakage and Evaluation Issues in Micro-Expression Analysis](https://mlsanity.com/paper/2211.11425/) (2022)
- [Improve Knowledge Distillation via Label Revision and Data Selection](https://mlsanity.com/paper/2404.03693/) (2024)

## Abstract (arXiv)

Micro expression recognition (MER) is a very challenging area of research due to its intrinsic nature and fine-grained changes. In the literature, the problem of MER has been solved through handcrafted/descriptor-based techniques. However, in recent times, deep learning (DL) based techniques have been adopted to gain higher performance for MER. Also, rich survey articles on MER are available by summarizing the datasets, experimental settings, conventional and deep learning methods. In contrast, these studies lack the ability to convey the impact of network design paradigms and experimental setting strategies for DL-based MER. Therefore, this paper aims to provide a deep insight into the DL-based MER frameworks with a perspective on promises in network model designing, experimental strategies, challenges, and research needs. Also, the detailed categorization of available MER frameworks is presented in various aspects of model design and technical characteristics. Moreover, an empirical analysis of the experimental and validation protocols adopted by MER methods is presented. The challenges mentioned earlier and network design strategies may assist the affective computing research community in forging ahead in MER research. Finally, we point out the future directions, research needs, and draw our conclusions.