# Facial Expression Classification using Fusion of Deep Neural Network in Video for the 3rd ABAW3 Competition

arXiv: https://arxiv.org/abs/2203.12899 · HTML: https://mlsanity.com/paper/2203.12899/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 24 Mar 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace](https://mlsanity.com/paper/1910.04855/) (2019)
- [Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond](https://mlsanity.com/paper/1804.10938/) (2018)
- [Analysing Affective Behavior in the First ABAW 2020 Competition](https://mlsanity.com/paper/2001.11409/) (2020)
- [Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework](https://mlsanity.com/paper/2103.15792/) (2021)
- [Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study](https://mlsanity.com/paper/2105.03790/) (2021)
- [Analysing Affective Behavior in the second ABAW2 Competition](https://mlsanity.com/paper/2106.15318/) (2021)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challenges](https://mlsanity.com/paper/2202.10659/) (2022)
- [Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network](https://mlsanity.com/paper/1910.11111/) (2019)
- [Transformer-based Multimodal Information Fusion for Facial Expression Analysis](https://mlsanity.com/paper/2203.12367/) (2022)
- [A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognition](https://mlsanity.com/paper/2107.04187/) (2021)
- [Iterative Distillation for Better Uncertainty Estimates in Multitask Emotion Recognition](https://mlsanity.com/paper/2108.04228/) (2021)
- [A Multi-task Mean Teacher for Semi-supervised Facial Affective Behavior Analysis](https://mlsanity.com/paper/2107.04225/) (2021)
- [Affect Expression Behaviour Analysis in the Wild using Spatio-Channel Attention and Complementary Context Information](https://mlsanity.com/paper/2009.14440/) (2020)
- [Emotion Recognition with Incomplete Labels Using Modified Multi-task Learning Technique](https://mlsanity.com/paper/2107.04192/) (2021)

## Built on by (most cited first, 5 of 5)

- [ABAW: Learning from Synthetic Data & Multi-Task Learning Challenges](https://mlsanity.com/paper/2207.01138/) (2022)
- [The 6th Affective Behavior Analysis in-the-wild (ABAW) Competition](https://mlsanity.com/paper/2402.19344/) (2024)
- [Multi-modal Facial Affective Analysis based on Masked Autoencoder](https://mlsanity.com/paper/2303.10849/) (2023)
- [Multimodal Feature Extraction and Fusion for Emotional Reaction Intensity Estimation and Expression Classification in Videos with Transformers](https://mlsanity.com/paper/2303.09164/) (2023)
- [Multimodal Fusion with Pre-Trained Model Features in Affective Behaviour Analysis In-the-wild](https://mlsanity.com/paper/2403.15044/) (2024)

## Abstract (arXiv)

For computers to recognize human emotions, expression classification is an equally important problem in the human-computer interaction area. In the 3rd Affective Behavior Analysis In-The-Wild competition, the task of expression classification includes eight classes with six basic expressions of human faces from videos. In this paper, we employ a transformer mechanism to encode the robust representation from the backbone. Fusion of the robust representations plays an important role in the expression classification task. Our approach achieves 30.35% and 28.60% for the F1 score on the validation set and the test set, respectively. This result shows the effectiveness of the proposed architecture based on the Aff-Wild2 dataset.