# Disentangling Identity and Pose for Facial Expression Recognition

arXiv: https://arxiv.org/abs/2208.08106 · HTML: https://mlsanity.com/paper/2208.08106/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 49 references that are in the map.
- First posted on arXiv 17 Aug 2022.

## Builds on (most cited first, 18 of 18)

- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Learning Face Representation from Scratch](https://mlsanity.com/paper/1411.7923/) (2014)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [Deep Face Recognition: A Survey](https://mlsanity.com/paper/1804.06655/) (2018)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distribution](https://mlsanity.com/paper/1608.01041/) (2016)
- [Suppressing Uncertainties for Large-Scale Facial Expression Recognition](https://mlsanity.com/paper/2002.10392/) (2020)
- [Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition](https://mlsanity.com/paper/1905.04075/) (2019)
- [Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition](https://mlsanity.com/paper/2104.00232/) (2021)
- [Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition](https://mlsanity.com/paper/2104.05160/) (2021)
- [Spatial-Temporal Recurrent Neural Network for Emotion Recognition](https://mlsanity.com/paper/1705.04515/) (2017)
- [Covariance Pooling For Facial Expression Recognition](https://mlsanity.com/paper/1805.04855/) (2018)
- [Island Loss for Learning Discriminative Features in Facial Expression Recognition](https://mlsanity.com/paper/1710.03144/) (2017)
- [FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition](https://mlsanity.com/paper/1609.06591/) (2016)
- [Local Learning with Deep and Handcrafted Features for Facial Expression Recognition](https://mlsanity.com/paper/1804.10892/) (2018)
- [Identity-Free Facial Expression Recognition using conditional Generative Adversarial Network](https://mlsanity.com/paper/1903.08051/) (2019)
- [Boosting Facial Expression Recognition by A Semi-Supervised Progressive Teacher](https://mlsanity.com/paper/2205.14361/) (2022)

## Built on by (most cited first, 11 of 11)

- [GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognition](https://mlsanity.com/paper/2312.04293/) (2023)
- [Disentangled Source-Free Personalization for Facial Expression Recognition with Neutral Target Data](https://mlsanity.com/paper/2503.20771/) (2025)
- [FER-former: Multi-modal Transformer for Facial Expression Recognition](https://mlsanity.com/paper/2303.12997/) (2023)
- [MuSACo: Multimodal Subject-Specific Selection and Adaptation for Expression Recognition with Co-Training](https://mlsanity.com/paper/2508.12522/) (2025)
- [FROQ: Observing Face Recognition Models for Efficient Quality Assessment](https://mlsanity.com/paper/2509.17689/) (2025)
- [Knowledge-Enhanced Facial Expression Recognition with Emotional-to-Neutral Transformation](https://mlsanity.com/paper/2409.08598/) (2024)
- [SLYKLatent: A Learning Framework for Gaze Estimation Using Deep Facial Feature Learning](https://mlsanity.com/paper/2402.01555/) (2024)
- [Reverse Personalization](https://mlsanity.com/paper/2512.22984/) (2025)
- [DrFER: Learning Disentangled Representations for 3D Facial Expression Recognition](https://mlsanity.com/paper/2403.08318/) (2024)
- [Ada-DF: An Adaptive Label Distribution Fusion Network For Facial Expression Recognition](https://mlsanity.com/paper/2404.15714/) (2024)
- [Emotion Knowledge Enhancement for Vision Large Language Models: A Self-Verification Approach for High-Quality Emotion Instruction Data Generation](https://mlsanity.com/paper/2505.18168/) (2025)

## Abstract (arXiv)

Facial expression recognition (FER) is a challenging problem because the expression component is always entangled with other irrelevant factors, such as identity and head pose. In this work, we propose an identity and pose disentangled facial expression recognition (IPD-FER) model to learn more discriminative feature representation. We regard the holistic facial representation as the combination of identity, pose and expression. These three components are encoded with different encoders. For identity encoder, a well pre-trained face recognition model is utilized and fixed during training, which alleviates the restriction on specific expression training data in previous works and makes the disentanglement practicable on in-the-wild datasets. At the same time, the pose and expression encoder are optimized with corresponding labels. Combining identity and pose feature, a neutral face of input individual should be generated by the decoder. When expression feature is added, the input image should be reconstructed. By comparing the difference between synthesized neutral and expressional images of the same individual, the expression component is further disentangled from identity and pose. Experimental results verify the effectiveness of our method on both lab-controlled and in-the-wild databases and we achieve state-of-the-art recognition performance.