# FaceTopoNet: Facial Expression Recognition using Face Topology Learning

arXiv: https://arxiv.org/abs/2209.06322 · HTML: https://mlsanity.com/paper/2209.06322/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 101 references that are in the map.
- First posted on arXiv 13 Sep 2022.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild](https://mlsanity.com/paper/1708.03985/) (2017)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [Suppressing Uncertainties for Large-Scale Facial Expression Recognition](https://mlsanity.com/paper/2002.10392/) (2020)
- [Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition](https://mlsanity.com/paper/1905.04075/) (2019)
- [Graph-Structured Representations for Visual Question Answering](https://mlsanity.com/paper/1609.05600/) (2016)
- [Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units](https://mlsanity.com/paper/1603.05201/) (2016)
- [TransFER: Learning Relation-aware Facial Expression Representations with Transformers](https://mlsanity.com/paper/2108.11116/) (2021)
- [Reasoning about Entailment with Neural Attention](https://mlsanity.com/paper/1509.06664/) (2015)
- [From Facial Expression Recognition to Interpersonal Relation Prediction](https://mlsanity.com/paper/1609.06426/) (2016)
- [Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition](https://mlsanity.com/paper/2104.05160/) (2021)
- [3D Face Morphable Models "In-the-Wild"](https://mlsanity.com/paper/1701.05360/) (2017)
- [Deep Learning using Linear Support Vector Machines](https://mlsanity.com/paper/1306.0239/) (2013)
- [Self-supervised Contrastive Learning of Multi-view Facial Expressions](https://mlsanity.com/paper/2108.06723/) (2021)
- [Facial Motion Prior Networks for Facial Expression Recognition](https://mlsanity.com/paper/1902.08788/) (2019)
- [Temporal Multimodal Fusion for Video Emotion Classification in the Wild](https://mlsanity.com/paper/1709.07200/) (2017)
- [Identity-Free Facial Expression Recognition using conditional Generative Adversarial Network](https://mlsanity.com/paper/1903.08051/) (2019)
- [BReG-NeXt: Facial Affect Computing Using Adaptive Residual Networks With Bounded Gradient](https://mlsanity.com/paper/2004.08495/) (2020)
- [Video-based Facial Expression Recognition using Graph Convolutional Networks](https://mlsanity.com/paper/2010.13386/) (2020)
- [Adversarial Graph Representation Adaptation for Cross-Domain Facial Expression Recognition](https://mlsanity.com/paper/2008.00859/) (2020)
- [Exploiting Emotional Dependencies with Graph Convolutional Networks for Facial Expression Recognition](https://mlsanity.com/paper/2106.03487/) (2021)
- [CapsField: Light Field-based Face and Expression Recognition in the Wild using Capsule Routing](https://mlsanity.com/paper/2101.03503/) (2021)
- [Expression Analysis Based on Face Regions in Read-world Conditions](https://mlsanity.com/paper/1911.05188/) (2019)
- [Facial Expression Recognition using Visual Saliency and Deep Learning](https://mlsanity.com/paper/1708.08016/) (2017)

## Built on by (most cited first, 4 of 4)

- [AffectNet+: A Database for Enhancing Facial Expression Recognition with Soft-Labels](https://mlsanity.com/paper/2410.22506/) (2024)
- [Toward Fair Facial Expression Recognition with Improved Distribution Alignment](https://mlsanity.com/paper/2306.06696/) (2023)
- [Exploring the Boundaries of Semi-Supervised Facial Expression Recognition using In-Distribution, Out-of-Distribution, and Unconstrained Data](https://mlsanity.com/paper/2306.01229/) (2023)
- [Active Learning with Contrastive Pre-training for Facial Expression Recognition](https://mlsanity.com/paper/2307.02744/) (2023)

## Abstract (arXiv)

Prior work has shown that the order in which different components of the face are learned using a sequential learner can play an important role in the performance of facial expression recognition systems. We propose FaceTopoNet, an end-to-end deep model for facial expression recognition, which is capable of learning an effective tree topology of the face. Our model then traverses the learned tree to generate a sequence, which is then used to form an embedding to feed a sequential learner. The devised model adopts one stream for learning structure and one stream for learning texture. The structure stream focuses on the positions of the facial landmarks, while the main focus of the texture stream is on the patches around the landmarks to learn textural information. We then fuse the outputs of the two streams by utilizing an effective attention-based fusion strategy. We perform extensive experiments on four large-scale in-the-wild facial expression datasets - namely AffectNet, FER2013, ExpW, and RAF-DB - and one lab-controlled dataset (CK+) to evaluate our approach. FaceTopoNet achieves state-of-the-art performance on three of the five datasets and obtains competitive results on the other two datasets. We also perform rigorous ablation and sensitivity experiments to evaluate the impact of different components and parameters in our model. Lastly, we perform robustness experiments and demonstrate that FaceTopoNet is more robust against occlusions in comparison to other leading methods in the area.