# HTNet for micro-expression recognition

arXiv: https://arxiv.org/abs/2307.14637 · HTML: https://mlsanity.com/paper/2307.14637/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 47 references that are in the map.
- First posted on arXiv 27 Jul 2023.

## Builds on (most cited first, 12 of 12)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Going Deeper in Spiking Neural Networks: VGG and Residual Architectures](https://mlsanity.com/paper/1802.02627/) (2018)
- [Nested Hierarchical Transformer: Towards Accurate, Data-Efficient and Interpretable Visual Understanding](https://mlsanity.com/paper/2105.12723/) (2021)
- [FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in Videos](https://mlsanity.com/paper/2203.09463/) (2022)
- [OFF-ApexNet on Micro-expression Recognition System](https://mlsanity.com/paper/1805.08699/) (2018)
- [Shallow Triple Stream Three-dimensional CNN (STSTNet) for Micro-expression Recognition](https://mlsanity.com/paper/1902.03634/) (2019)
- [Less is More: Micro-expression Recognition from Video using Apex Frame](https://mlsanity.com/paper/1606.01721/) (2016)
- [Revealing the Invisible with Model and Data Shrinking for Composite-database Micro-expression Recognition](https://mlsanity.com/paper/2006.09674/) (2020)
- [Spatiotemporal Recurrent Convolutional Networks for Recognizing Spontaneous Micro-expressions](https://mlsanity.com/paper/1901.04656/) (2019)
- [Feature refinement: An expression-specific feature learning and fusion method for micro-expression recognition](https://mlsanity.com/paper/2101.04838/) (2021)
- [Short and Long Range Relation Based Spatio-Temporal Transformer for Micro-Expression Recognition](https://mlsanity.com/paper/2112.05851/) (2021)

## Built on by (most cited first, 20 of 20)

- [ResEmoteNet: Bridging Accuracy and Loss Reduction in Facial Emotion Recognition](https://mlsanity.com/paper/2409.10545/) (2024)
- [Visual and Textual Prompts in VLLMs for Enhancing Emotion Recognition](https://mlsanity.com/paper/2504.17224/) (2025)
- [LRDif: Diffusion Models for Under-Display Camera Emotion Recognition](https://mlsanity.com/paper/2402.00250/) (2024)
- [Robust Dynamic Facial Expression Recognition](https://mlsanity.com/paper/2502.16129/) (2025)
- [SpotFormer: Multi-Scale Spatio-Temporal Transformer for Facial Expression Spotting](https://mlsanity.com/paper/2407.20799/) (2024)
- [MPFNet: A Multi-Prior Fusion Network with a Progressive Training Strategy for Micro-Expression Recognition](https://mlsanity.com/paper/2506.09735/) (2025)
- [MELLM: A Flow-Guided Large Language Model for Micro-Expression Understanding](https://mlsanity.com/paper/2505.07007/) (2025)
- [FED-PsyAU: Privacy-Preserving Micro-Expression Recognition via Psychological AU Coordination and Dynamic Facial Motion Modeling](https://mlsanity.com/paper/2507.20557/) (2025)
- [Rethinking Key-frame-based Micro-expression Recognition: A Robust and Accurate Framework Against Key-frame Errors](https://mlsanity.com/paper/2508.06640/) (2025)
- [Hierarchical Space-Time Attention for Micro-Expression Recognition](https://mlsanity.com/paper/2405.03202/) (2024)
- [Improving Facial Emotion Recognition through Dataset Merging and Balanced Training Strategies](https://mlsanity.com/paper/2604.20307/) (2026)
- [Three-Stream Temporal-Shift Attention Network Based on Self-Knowledge Distillation for Micro-Expression Recognition](https://mlsanity.com/paper/2406.17538/) (2024)
- [LLDif: Diffusion Models for Low-light Emotion Recognition](https://mlsanity.com/paper/2408.04235/) (2024)
- [AHMSA-Net: Adaptive Hierarchical Multi-Scale Attention Network for Micro-Expression Recognition](https://mlsanity.com/paper/2501.02539/) (2025)
- [MMME: A Spontaneous Multi-Modal Micro-Expression Dataset Enabling Visual-Physiological Fusion](https://mlsanity.com/paper/2506.09834/) (2025)
- [Dynamic Fusion-Aware Graph Convolutional Neural Network for Multimodal Emotion Recognition in Conversations](https://mlsanity.com/paper/2603.22345/) (2026)
- [EPIR: An Efficient Patch Tokenization, Integration and Representation Framework for Micro-expression Recognition](https://mlsanity.com/paper/2604.08106/) (2026)
- [MEDN: Motion-Emotion Feature Decoupling Network for Micro-Expression Recognition](https://mlsanity.com/paper/2604.17899/) (2026)
- [CDER-SME: A Cross-Device Event-RGB Micro-Expression Dataset under Multi-Level Stress Induction](https://mlsanity.com/paper/2606.20715/) (2026)
- [SAC2-Net: Semantic Anchoring and Complementary-Consensus Fusion for Multimodal Micro-Expression Recognition](https://mlsanity.com/paper/2606.25542/) (2026)

## Abstract (arXiv)

Facial expression is related to facial muscle contractions and different muscle movements correspond to different emotional states. For micro-expression recognition, the muscle movements are usually subtle, which has a negative impact on the performance of current facial emotion recognition algorithms. Most existing methods use self-attention mechanisms to capture relationships between tokens in a sequence, but they do not take into account the inherent spatial relationships between facial landmarks. This can result in sub-optimal performance on micro-expression recognition tasks.Therefore, learning to recognize facial muscle movements is a key challenge in the area of micro-expression recognition. In this paper, we propose a Hierarchical Transformer Network (HTNet) to identify critical areas of facial muscle movement. HTNet includes two major components: a transformer layer that leverages the local temporal features and an aggregation layer that extracts local and global semantical facial features. Specifically, HTNet divides the face into four different facial areas: left lip area, left eye area, right eye area and right lip area. The transformer layer is used to focus on representing local minor muscle movement with local self-attention in each area. The aggregation layer is used to learn the interactions between eye areas and lip areas. The experiments on four publicly available micro-expression datasets show that the proposed approach outperforms previous methods by a large margin. The codes and models are available at: urlhttps://github.com/wangzhifengharrison/HTNet