# MOL: Joint Estimation of Micro-Expression, Optical Flow, and Landmark via Transformer-Graph-Style Convolution

arXiv: https://arxiv.org/abs/2506.14511 · HTML: https://mlsanity.com/paper/2506.14511/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 72 references that are in the map.
- First posted on arXiv 17 Jun 2025.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Spectral Networks and Locally Connected Networks on Graphs](https://mlsanity.com/paper/1312.6203/) (2013)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace](https://mlsanity.com/paper/1910.04855/) (2019)
- [Analysing Affective Behavior in the second ABAW2 Competition](https://mlsanity.com/paper/2106.15318/) (2021)
- [Interpretable Convolutional Neural Networks](https://mlsanity.com/paper/1710.00935/) (2017)
- [Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness](https://mlsanity.com/paper/1608.05842/) (2016)
- [JhatAA-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attention](https://mlsanity.com/paper/2003.08834/) (2020)
- [Video-based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms](https://mlsanity.com/paper/2201.12728/) (2022)
- [Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning](https://mlsanity.com/paper/2008.00923/) (2020)
- [OFF-ApexNet on Micro-expression Recognition System](https://mlsanity.com/paper/1805.08699/) (2018)
- [Shallow Triple Stream Three-dimensional CNN (STSTNet) for Micro-expression Recognition](https://mlsanity.com/paper/1902.03634/) (2019)
- [Towards Reading Hidden Emotions: A comparative Study of Spontaneous Micro-expression Spotting and Recognition Methods](https://mlsanity.com/paper/1511.00423/) (2015)
- [Learning Deep Representation for Face Alignment with Auxiliary Attributes](https://mlsanity.com/paper/1408.3967/) (2014)
- [Less is More: Micro-expression Recognition from Video using Apex Frame](https://mlsanity.com/paper/1606.01721/) (2016)
- [AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectors](https://mlsanity.com/paper/2403.04697/) (2024)
- [Enriched Long-term Recurrent Convolutional Network for Facial Micro-Expression Recognition](https://mlsanity.com/paper/1805.08417/) (2018)
- [LEARNet Dynamic Imaging Network for Micro Expression Recognition](https://mlsanity.com/paper/1904.09410/) (2019)
- [MER-GCN: Micro Expression Recognition Based on Relation Modeling with Graph Convolutional Network](https://mlsanity.com/paper/2004.08915/) (2020)
- [Spontaneous Facial Micro-Expression Recognition using 3D Spatiotemporal Convolutional Neural Networks](https://mlsanity.com/paper/1904.01390/) (2019)
- [Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition](https://mlsanity.com/paper/2112.05851/) (2021)
- [Feature Representation Learning with Adaptive Displacement Generation and Transformer Fusion for Micro-Expression Recognition](https://mlsanity.com/paper/2304.04420/) (2023)
- [Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection](https://mlsanity.com/paper/2308.07770/) (2023)

## Built on by (most cited first, 12 of 12)

- [Hybrid Generative Fusion for Efficient and Privacy-Preserving Face Recognition Dataset Generation](https://mlsanity.com/paper/2508.10672/) (2025)
- [RegFormer++: An Efficient Large-Scale 3D LiDAR Point Registration Network with Projection-Aware 2D Transformer](https://mlsanity.com/paper/2603.14290/) (2026)
- [Evaluating and Correcting Human Annotation Bias in Dynamic Micro-Expression Recognition](https://mlsanity.com/paper/2603.04766/) (2026)
- [ME-TST+: Micro-expression Analysis via Temporal State Transition with ROI Relationship Awareness](https://mlsanity.com/paper/2508.08082/) (2025)
- [Boosting Micro-Expression Analysis via Prior-Guided Video-Level Regression](https://mlsanity.com/paper/2508.18834/) (2025)
- [Divide, Weight, and Route: Difficulty-Aware Optimization with Dynamic Expert Fusion for Long-tailed Recognition](https://mlsanity.com/paper/2508.19630/) (2025)
- [Apex-Centered Spatio-Temporal Rank Pooling and Gradient Attention for Micro-Expression Recognition](https://mlsanity.com/paper/2509.00056/) (2025)
- [FMANet: A Novel Dual-Phase Optical Flow Approach with Fusion Motion Attention Network for Robust Micro-expression Recognition](https://mlsanity.com/paper/2510.07810/) (2025)
- [Adaptive Fusion Network with Temporal-Ranked and Motion-Intensity Dynamic Images for Micro-expression Recognition](https://mlsanity.com/paper/2510.09730/) (2025)
- [Micro-Expression-Aware Avatar Fingerprinting via Inter-Frame Feature Differencing](https://mlsanity.com/paper/2604.23247/) (2026)
- [SAC2-Net: Semantic Anchoring and Complementary-Consensus Fusion for Multimodal Micro-Expression Recognition](https://mlsanity.com/paper/2606.25542/) (2026)
- [STAG: Spatio-temporal Evolving Structural Representation of Action Units for Micro-expression Recognition](https://mlsanity.com/paper/2606.28083/) (2026)

## Abstract (arXiv)

Facial micro-expression recognition (MER) is a challenging problem, due to transient and subtle micro-expression (ME) actions. Most existing methods depend on hand-crafted features, key frames like onset, apex, and offset frames, or deep networks limited by small-scale and low-diversity datasets. In this paper, we propose an end-to-end micro-action-aware deep learning framework with advantages from transformer, graph convolution, and vanilla convolution. In particular, we propose a novel F5C block composed of fully-connected convolution and channel correspondence convolution to directly extract local-global features from a sequence of raw frames, without the prior knowledge of key frames. The transformer-style fully-connected convolution is proposed to extract local features while maintaining global receptive fields, and the graph-style channel correspondence convolution is introduced to model the correlations among feature patterns. Moreover, MER, optical flow estimation, and facial landmark detection are jointly trained by sharing the local-global features. The two latter tasks contribute to capturing facial subtle action information for MER, which can alleviate the impact of insufficient training data. Extensive experiments demonstrate that our framework (i) outperforms the state-of-the-art MER methods on CASME II, SAMM, and SMIC benchmarks, (ii) works well for optical flow estimation and facial landmark detection, and (iii) can capture facial subtle muscle actions in local regions associated with MEs. The code is available at https://github.com/CYF-cuber/MOL.