# 3D-CNN for Facial Micro- and Macro-expression Spotting on Long Video Sequences using Temporal Oriented Reference Frame

arXiv: https://arxiv.org/abs/2105.06340 · HTML: https://mlsanity.com/paper/2105.06340/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 36 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 9 of 9)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [SAMM Long Videos: A Spontaneous Facial Micro- and Macro-Expressions Dataset](https://mlsanity.com/paper/1911.01519/) (2019)
- [Micro-expression detection in long videos using optical flow and recurrent neural networks](https://mlsanity.com/paper/1903.10765/) (2019)
- [Objective Classes for Micro-Facial Expression Recognition](https://mlsanity.com/paper/1708.07549/) (2017)
- [Spotting Macro- and Micro-expression Intervals in Long Video Sequences](https://mlsanity.com/paper/1912.11985/) (2019)
- [ADS-ME: Anomaly Detection System for Micro-expression Spotting](https://mlsanity.com/paper/1903.04354/) (2019)

## Built on by (most cited first, 16 of 16)

- [AU-aware graph convolutional network for Macro- and Micro-expression spotting](https://mlsanity.com/paper/2303.09114/) (2023)
- [Synergistic Spotting and Recognition of Micro-Expression via Temporal State Transition](https://mlsanity.com/paper/2409.09707/) (2024)
- [Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Survey](https://mlsanity.com/paper/2412.17616/) (2024)
- [Weakly-supervised Micro- and Macro-expression Spotting Based on Multi-level Consistency](https://mlsanity.com/paper/2305.02734/) (2023)
- [Multi-Scale Spatio-Temporal Graph Convolutional Network for Facial Expression Spotting](https://mlsanity.com/paper/2403.15994/) (2024)
- [Exploring Timeline Control for Facial Motion Generation](https://mlsanity.com/paper/2505.20861/) (2025)
- [RMES: Real-Time Micro-Expression Spotting Using Phase From Riesz Pyramid](https://mlsanity.com/paper/2305.05523/) (2023)
- [SpotFormer: Multi-Scale Spatio-Temporal Transformer for Facial Expression Spotting](https://mlsanity.com/paper/2407.20799/) (2024)
- [Causal-Ex: Causal Graph-based Micro and Macro Expression Spotting](https://mlsanity.com/paper/2503.09098/) (2025)
- [Weak Supervision with Arbitrary Single Frame for Micro- and Macro-expression Spotting](https://mlsanity.com/paper/2403.14240/) (2024)
- [MEGC2025: Micro-Expression Grand Challenge on Spot Then Recognize and Visual Question Answering](https://mlsanity.com/paper/2506.15298/) (2025)
- [ME-TST+: Micro-expression Analysis via Temporal State Transition with ROI Relationship Awareness](https://mlsanity.com/paper/2508.08082/) (2025)
- [Boosting Micro-Expression Analysis via Prior-Guided Video-Level Regression](https://mlsanity.com/paper/2508.18834/) (2025)
- [PESFormer: Boosting Macro- and Micro-expression Spotting with Direct Timestamp Encoding](https://mlsanity.com/paper/2410.18695/) (2024)
- [Point-Supervised Facial Expression Spotting with Gaussian-Based Instance-Adaptive Intensity Modeling](https://mlsanity.com/paper/2511.16952/) (2025)
- [MEGC2026: Micro-Expression Grand Challenge on Visual Question Answering](https://mlsanity.com/paper/2603.08927/) (2026)

## Abstract (arXiv)

Facial expression spotting is the preliminary step for micro- and macro-expression analysis. The task of reliably spotting such expressions in video sequences is currently unsolved. The current best systems depend upon optical flow methods to extract regional motion features, before categorisation of that motion into a specific class of facial movement. Optical flow is susceptible to drift error, which introduces a serious problem for motions with long-term dependencies, such as high frame-rate macro-expression. We propose a purely deep learning solution which, rather than tracking frame differential motion, compares via a convolutional model, each frame with two temporally local reference frames. Reference frames are sampled according to calculated micro- and macro-expression duration. As baseline for MEGC2021 using leave-one-subject-out evaluation method, we show that our solution achieves F1-score of 0.105 in a high frame-rate (200 fps) SAMM long videos dataset (SAMM-LV) and is competitive in a low frame-rate (30 fps) (CAS(ME)2) dataset. On unseen MEGC2022 challenge dataset, the baseline results are 0.1176 on SAMM Challenge dataset, 0.1739 on CAS(ME)3 and overall performance of 0.1531 on both dataset.