# Feature Representation Learning with Adaptive Displacement Generation and Transformer Fusion for Micro-Expression Recognition

arXiv: https://arxiv.org/abs/2304.04420 · HTML: https://mlsanity.com/paper/2304.04420/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 56 references that are in the map.
- First posted on arXiv 10 Apr 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms](https://mlsanity.com/paper/2201.12728/) (2022)
- [Learning-based Video Motion Magnification](https://mlsanity.com/paper/1804.02684/) (2018)
- [Deep Learning for Micro-expression Recognition: A Survey](https://mlsanity.com/paper/2107.02823/) (2021)
- [Shallow Triple Stream Three-dimensional CNN (STSTNet) for Micro-expression Recognition](https://mlsanity.com/paper/1902.03634/) (2019)
- [Towards Reading Hidden Emotions: A comparative Study of Spontaneous Micro-expression Spotting and Recognition Methods](https://mlsanity.com/paper/1511.00423/) (2015)
- [Less is More: Micro-expression Recognition from Video using Apex Frame](https://mlsanity.com/paper/1606.01721/) (2016)
- [Revealing the Invisible with Model and Data Shrinking for Composite-database Micro-expression Recognition](https://mlsanity.com/paper/2006.09674/) (2020)
- [Enriched Long-term Recurrent Convolutional Network for Facial Micro-Expression Recognition](https://mlsanity.com/paper/1805.08417/) (2018)
- [Dual Graph Convolutional Networks with Transformer and Curriculum Learning for Image Captioning](https://mlsanity.com/paper/2108.02366/) (2021)
- [Spatiotemporal Recurrent Convolutional Networks for Recognizing Spontaneous Micro-expressions](https://mlsanity.com/paper/1901.04656/) (2019)
- [Feature refinement: An expression-specific feature learning and fusion method for micro-expression recognition](https://mlsanity.com/paper/2101.04838/) (2021)
- [LEARNet Dynamic Imaging Network for Micro Expression Recognition](https://mlsanity.com/paper/1904.09410/) (2019)
- [Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition](https://mlsanity.com/paper/2112.05851/) (2021)
- [AU R-CNN: Encoding Expert Prior Knowledge into R-CNN for Action Unit Detection](https://mlsanity.com/paper/1812.05788/) (2018)
- [An Overview of Facial Micro-Expression Analysis: Data, Methodology and Challenge](https://mlsanity.com/paper/2012.11307/) (2020)
- [Graph-based Facial Affect Analysis: A Review](https://mlsanity.com/paper/2103.15599/) (2021)
- [Cross-Database Micro-Expression Recognition: A Benchmark](https://mlsanity.com/paper/1812.07742/) (2018)
- [AffectiveNet: Affective-Motion Feature Learningfor Micro Expression Recognition](https://mlsanity.com/paper/2104.07569/) (2021)
- [ICE-GAN: Identity-aware and Capsule-Enhanced GAN with Graph-based Reasoning for Micro-Expression Recognition and Synthesis](https://mlsanity.com/paper/2005.04370/) (2020)

## Built on by (most cited first, 17 of 17)

- [MOL: Joint Estimation of Micro-Expression, Optical Flow, and Landmark via Transformer-Graph-Style Convolution](https://mlsanity.com/paper/2506.14511/) (2025)
- [Disentangled Representation Learning for Controllable Person Image Generation](https://mlsanity.com/paper/2312.05798/) (2023)
- [Action Unit Enhance Dynamic Facial Expression Recognition](https://mlsanity.com/paper/2507.07678/) (2025)
- [FED-PsyAU: Privacy-Preserving Micro-Expression Recognition via Psychological AU Coordination and Dynamic Facial Motion Modeling](https://mlsanity.com/paper/2507.20557/) (2025)
- [Evaluating and Correcting Human Annotation Bias in Dynamic Micro-Expression Recognition](https://mlsanity.com/paper/2603.04766/) (2026)
- [Hierarchical Space-Time Attention for Micro-Expression Recognition](https://mlsanity.com/paper/2405.03202/) (2024)
- [Temporal and Spatial Feature Fusion Framework for Dynamic Micro Expression Recognition](https://mlsanity.com/paper/2505.16372/) (2025)
- [Apex-Centered Spatio-Temporal Rank Pooling and Gradient Attention for Micro-Expression Recognition](https://mlsanity.com/paper/2509.00056/) (2025)
- [A Benchmark for Incremental Micro-expression Recognition](https://mlsanity.com/paper/2501.19111/) (2025)
- [FaceSleuth-R: Adaptive Orientation-Aware Attention for Robust Micro-Expression Recognition](https://mlsanity.com/paper/2506.02695/) (2025)
- [MPT: Motion Prompt Tuning for Micro-Expression Recognition](https://mlsanity.com/paper/2508.09446/) (2025)
- [Pretrained Diffusion Models Are Inherently Skipped-Step Samplers](https://mlsanity.com/paper/2508.15233/) (2025)
- [FMANet: A Novel Dual-Phase Optical Flow Approach with Fusion Motion Attention Network for Robust Micro-expression Recognition](https://mlsanity.com/paper/2510.07810/) (2025)
- [Point-Supervised Facial Expression Spotting with Gaussian-Based Instance-Adaptive Intensity Modeling](https://mlsanity.com/paper/2511.16952/) (2025)
- [EPIR: An Efficient Patch Tokenization, Integration and Representation Framework for Micro-expression Recognition](https://mlsanity.com/paper/2604.08106/) (2026)
- [MEDN: Motion-Emotion Feature Decoupling Network for Micro-Expression Recognition](https://mlsanity.com/paper/2604.17899/) (2026)
- [SAC2-Net: Semantic Anchoring and Complementary-Consensus Fusion for Multimodal Micro-Expression Recognition](https://mlsanity.com/paper/2606.25542/) (2026)

## Abstract (arXiv)

Micro-expressions are spontaneous, rapid and subtle facial movements that can neither be forged nor suppressed. They are very important nonverbal communication clues, but are transient and of low intensity thus difficult to recognize. Recently deep learning based methods have been developed for micro-expression (ME) recognition using feature extraction and fusion techniques, however, targeted feature learning and efficient feature fusion still lack further study according to the ME characteristics. To address these issues, we propose a novel framework Feature Representation Learning with adaptive Displacement Generation and Transformer fusion (FRL-DGT), in which a convolutional Displacement Generation Module (DGM) with self-supervised learning is used to extract dynamic features from onset/apex frames targeted to the subsequent ME recognition task, and a well-designed Transformer Fusion mechanism composed of three Transformer-based fusion modules (local, global fusions based on AU regions and full-face fusion) is applied to extract the multi-level informative features after DGM for the final ME prediction. The extensive experiments with solid leave-one-subject-out (LOSO) evaluation results have demonstrated the superiority of our proposed FRL-DGT to state-of-the-art methods.