# Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition

arXiv: https://arxiv.org/abs/2112.05851 · HTML: https://mlsanity.com/paper/2112.05851/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 88 references that are in the map.
- First posted on arXiv 10 Dec 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [VideoBERT: A Joint Model for Video and Language Representation Learning](https://mlsanity.com/paper/1904.01766/) (2019)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [Learning Deep Transformer Models for Machine Translation](https://mlsanity.com/paper/1906.01787/) (2019)
- [Cross-Modal Self-Attention Network for Referring Image Segmentation](https://mlsanity.com/paper/1904.04745/) (2019)
- [Adaptive Input Representations for Neural Language Modeling](https://mlsanity.com/paper/1809.10853/) (2018)
- [Facial Expression Recognition with Visual Transformers and Attentional Selective Fusion](https://mlsanity.com/paper/2103.16854/) (2021)
- [OFF-ApexNet on Micro-expression Recognition System](https://mlsanity.com/paper/1805.08699/) (2018)
- [Shallow Triple Stream Three-dimensional CNN (STSTNet) for Micro-expression Recognition](https://mlsanity.com/paper/1902.03634/) (2019)
- [Towards Reading Hidden Emotions: A comparative Study of Spontaneous Micro-expression Spotting and Recognition Methods](https://mlsanity.com/paper/1511.00423/) (2015)
- [Less is More: Micro-expression Recognition from Video using Apex Frame](https://mlsanity.com/paper/1606.01721/) (2016)
- [Revealing the Invisible with Model and Data Shrinking for Composite-database Micro-expression Recognition](https://mlsanity.com/paper/2006.09674/) (2020)
- [Enriched Long-term Recurrent Convolutional Network for Facial Micro-Expression Recognition](https://mlsanity.com/paper/1805.08417/) (2018)
- [Spatiotemporal Recurrent Convolutional Networks for Recognizing Spontaneous Micro-expressions](https://mlsanity.com/paper/1901.04656/) (2019)
- [MER-GCN: Micro Expression Recognition Based on Relation Modeling with Graph Convolutional Network](https://mlsanity.com/paper/2004.08915/) (2020)
- [Spontaneous Facial Micro-Expression Recognition using 3D Spatiotemporal Convolutional Neural Networks](https://mlsanity.com/paper/1904.01390/) (2019)
- [Micro-Attention for Micro-Expression recognition](https://mlsanity.com/paper/1811.02360/) (2018)
- [A Novel Apex-Time Network for Cross-Dataset Micro-Expression Recognition](https://mlsanity.com/paper/1904.03699/) (2019)

## Built on by (most cited first, 18 of 18)

- [Benchmarking Micro-action Recognition: Dataset, Methods, and Applications](https://mlsanity.com/paper/2403.05234/) (2024)
- [HTNet for micro-expression recognition](https://mlsanity.com/paper/2307.14637/) (2023)
- [Feature Representation Learning with Adaptive Displacement Generation and Transformer Fusion for Micro-Expression Recognition](https://mlsanity.com/paper/2304.04420/) (2023)
- [MOL: Joint Estimation of Micro-Expression, Optical Flow, and Landmark via Transformer-Graph-Style Convolution](https://mlsanity.com/paper/2506.14511/) (2025)
- [Data Leakage and Evaluation Issues in Micro-Expression Analysis](https://mlsanity.com/paper/2211.11425/) (2022)
- [Learning to Rank Onset-Occurring-Offset Representations for Micro-Expression Recognition](https://mlsanity.com/paper/2310.04664/) (2023)
- [MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection](https://mlsanity.com/paper/2204.01349/) (2022)
- [MPFNet: A Multi-Prior Fusion Network with a Progressive Training Strategy for Micro-Expression Recognition](https://mlsanity.com/paper/2506.09735/) (2025)
- [Adaptive Temporal Motion Guided Graph Convolution Network for Micro-expression Recognition](https://mlsanity.com/paper/2406.08997/) (2024)
- [Rethinking Key-frame-based Micro-expression Recognition: A Robust and Accurate Framework Against Key-frame Errors](https://mlsanity.com/paper/2508.06640/) (2025)
- [Multi-scale multi-modal micro-expression recognition algorithm based on transformer](https://mlsanity.com/paper/2301.02969/) (2023)
- [Multimodal Latent Emotion Recognition from Micro-expression and Physiological Signals](https://mlsanity.com/paper/2308.12156/) (2023)
- [A Benchmark for Incremental Micro-expression Recognition](https://mlsanity.com/paper/2501.19111/) (2025)
- [MMME: A Spontaneous Multi-Modal Micro-Expression Dataset Enabling Visual-Physiological Fusion](https://mlsanity.com/paper/2506.09834/) (2025)
- [MPT: Motion Prompt Tuning for Micro-Expression Recognition](https://mlsanity.com/paper/2508.09446/) (2025)
- [Micro-expression Recognition Based on Dual-branch Feature Extraction and Fusion](https://mlsanity.com/paper/2602.23950/) (2026)
- [Micro-AU CLIP: Fine-Grained Contrastive Learning from Local Independence to Global Dependency for Micro-Expression Action Unit Detection](https://mlsanity.com/paper/2603.16302/) (2026)
- [EPIR: An Efficient Patch Tokenization, Integration and Representation Framework for Micro-expression Recognition](https://mlsanity.com/paper/2604.08106/) (2026)

## Abstract (arXiv)

Being spontaneous, micro-expressions are useful in the inference of a person's true emotions even if an attempt is made to conceal them. Due to their short duration and low intensity, the recognition of micro-expressions is a difficult task in affective computing. The early work based on handcrafted spatio-temporal features which showed some promise, has recently been superseded by different deep learning approaches which now compete for the state of the art performance. Nevertheless, the problem of capturing both local and global spatio-temporal patterns remains challenging. To this end, herein we propose a novel spatio-temporal transformer architecture -- to the best of our knowledge, the first purely transformer based approach (i.e. void of any convolutional network use) for micro-expression recognition. The architecture comprises a spatial encoder which learns spatial patterns, a temporal aggregator for temporal dimension analysis, and a classification head. A comprehensive evaluation on three widely used spontaneous micro-expression data sets, namely SMIC-HS, CASME II and SAMM, shows that the proposed approach consistently outperforms the state of the art, and is the first framework in the published literature on micro-expression recognition to achieve the unweighted F1-score greater than 0.9 on any of the aforementioned data sets.