# Action Unit Enhance Dynamic Facial Expression Recognition

arXiv: https://arxiv.org/abs/2507.07678 · HTML: https://mlsanity.com/paper/2507.07678/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 45 references that are in the map.
- First posted on arXiv 10 Jul 2025.

## Builds on (most cited first, 18 of 18)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [DFEW: A Large-Scale Database for Recognizing Dynamic Facial Expressions in the Wild](https://mlsanity.com/paper/2008.05924/) (2020)
- [Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition](https://mlsanity.com/paper/1811.07770/) (2018)
- [FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in Videos](https://mlsanity.com/paper/2203.09463/) (2022)
- [MAFW: A Large-scale, Multi-modal, Compound Affective Database for Dynamic Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2208.00847/) (2022)
- [MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition](https://mlsanity.com/paper/2307.02227/) (2023)
- [Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2208.10335/) (2022)
- [From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos](https://mlsanity.com/paper/2312.05447/) (2023)
- [Spatio-Temporal Transformer for Dynamic Facial Expression Recognition in the Wild](https://mlsanity.com/paper/2205.04749/) (2022)
- [SVFAP: Self-supervised Video Facial Affect Perceiver](https://mlsanity.com/paper/2401.00416/) (2023)
- [Feature Representation Learning with Adaptive Displacement Generation and Transformer Fusion for Micro-Expression Recognition](https://mlsanity.com/paper/2304.04420/) (2023)
- [AU-Aware Vision Transformers for Biased Facial Expression Recognition](https://mlsanity.com/paper/2211.06609/) (2022)
- [FE-Adapter: Adapting Image-based Emotion Classifiers to Videos](https://mlsanity.com/paper/2408.02421/) (2024)
- [Robust Dynamic Facial Expression Recognition](https://mlsanity.com/paper/2502.16129/) (2025)

## Built on by (most cited first, 3 of 3)

- [Explainable graph attention network for stress recognition (StressGAT) via differential action units](https://mlsanity.com/paper/2607.20819/) (2026)
- [Why Do Vision Language Models Struggle To Recognize Human Emotions?](https://mlsanity.com/paper/2604.15280/) (2026)
- [Bidirectional Learning of Facial Action Units and Expressions via Structured Semantic Mapping across Heterogeneous Datasets](https://mlsanity.com/paper/2604.10541/) (2026)

## Abstract (arXiv)

Dynamic Facial Expression Recognition(DFER) is a rapidly evolving field of research that focuses on the recognition of time-series facial expressions. While previous research on DFER has concentrated on feature learning from a deep learning perspective, we put forward an AU-enhanced Dynamic Facial Expression Recognition architecture, namely AU-DFER, that incorporates AU-expression knowledge to enhance the effectiveness of deep learning modeling. In particular, the contribution of the Action Units(AUs) to different expressions is quantified, and a weight matrix is designed to incorporate a priori knowledge. Subsequently, the knowledge is integrated with the learning outcomes of a conventional deep learning network through the introduction of AU loss. The design is incorporated into the existing optimal model for dynamic expression recognition for the purpose of validation. Experiments are conducted on three recent mainstream open-source approaches to DFER on the principal datasets in this field. The results demonstrate that the proposed architecture outperforms the state-of-the-art(SOTA) methods without the need for additional arithmetic and generally produces improved results. Furthermore, we investigate the potential of AU loss function redesign to address data label imbalance issues in established dynamic expression datasets. To the best of our knowledge, this is the first attempt to integrate quantified AU-expression knowledge into various DFER models. We also devise strategies to tackle label imbalance, or minor class problems. Our findings suggest that employing a diverse strategy of loss function design can enhance the effectiveness of DFER. This underscores the criticality of addressing data imbalance challenges in mainstream datasets within this domain. The source code is available at https://github.com/Cross-Innovation-Lab/AU-DFER.