# ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challenges

arXiv: https://arxiv.org/abs/2303.01498 · HTML: https://mlsanity.com/paper/2303.01498/

## Key facts

- Cited by 110 papers in the map (data through 14 Oct 2026).
- Builds on 70 of its 116 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Builds on (most cited first, 30 of 70)

- [Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFace](https://mlsanity.com/paper/1910.04855/) (2019)
- [Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyond](https://mlsanity.com/paper/1804.10938/) (2018)
- [Analysing Affective Behavior in the First ABAW 2020 Competition](https://mlsanity.com/paper/2001.11409/) (2020)
- [Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework](https://mlsanity.com/paper/2103.15792/) (2021)
- [Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study](https://mlsanity.com/paper/2105.03790/) (2021)
- [Analysing Affective Behavior in the second ABAW2 Competition](https://mlsanity.com/paper/2106.15318/) (2021)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challenges](https://mlsanity.com/paper/2202.10659/) (2022)
- [Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network](https://mlsanity.com/paper/1910.11111/) (2019)
- [ABAW: Learning from Synthetic Data & Multi-Task Learning Challenges](https://mlsanity.com/paper/2207.01138/) (2022)
- [Aff-Wild2: Extending the Aff-Wild Database for Affect Recognition](https://mlsanity.com/paper/1811.07770/) (2018)
- [Deep Transparent Prediction through Latent Representation Analysis](https://mlsanity.com/paper/2009.07044/) (2020)
- [MIA-COV19D: COVID-19 Detection through 3-D Chest CT Image Analysis](https://mlsanity.com/paper/2106.07524/) (2021)
- [Two-Stream Aural-Visual Affect Analysis in the Wild](https://mlsanity.com/paper/2002.03399/) (2020)
- [Transformer-based Multimodal Information Fusion for Facial Expression Analysis](https://mlsanity.com/paper/2203.12367/) (2022)
- [Exploiting multi-CNN features in CNN-RNN based Dimensional Emotion Recognition on the OMG in-the-wild Dataset](https://mlsanity.com/paper/1910.01417/) (2019)
- [Multitask Emotion Recognition with Incomplete Labels](https://mlsanity.com/paper/2002.03557/) (2020)
- [A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognition](https://mlsanity.com/paper/2203.14779/) (2022)
- [Photorealistic Facial Synthesis in the Dimensional Affect Space](https://mlsanity.com/paper/1811.08004/) (2018)
- [MixAugment & Mixup: Augmentation Methods for Facial Expression Recognition](https://mlsanity.com/paper/2205.04442/) (2022)
- [A Multi-Task Learning & Generation Framework: Valence-Arousal, Action Units & Primary Expressions](https://mlsanity.com/paper/1811.07771/) (2018)
- [A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognition](https://mlsanity.com/paper/2107.04187/) (2021)
- [The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stress](https://mlsanity.com/paper/2207.05691/) (2022)
- [Training Deep Neural Networks with Different Datasets In-the-wild: The Emotion Recognition Paradigm](https://mlsanity.com/paper/1809.04359/) (2018)
- [Continuous Emotion Recognition using Visual-audio-linguistic information: A Technical Report for ABAW3](https://mlsanity.com/paper/2203.13031/) (2022)
- [A Multi-task Mean Teacher for Semi-supervised Facial Affective Behavior Analysis](https://mlsanity.com/paper/2107.04225/) (2021)
- [Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competition](https://mlsanity.com/paper/2107.03708/) (2021)
- [Coarse-to-Fine Cascaded Networks with Smooth Predicting for Video Facial Expression Recognition](https://mlsanity.com/paper/2203.13052/) (2022)
- [Incremental Boosting Convolutional Neural Network for Facial Action Unit Recognition](https://mlsanity.com/paper/1707.05395/) (2017)
- [Frame-level Prediction of Facial Expressions, Valence, Arousal and Action Units for Mobile Devices](https://mlsanity.com/paper/2203.13436/) (2022)
- [MMA-MRNNet: Harnessing Multiple Models of Affect and Dynamic Masked RNN for Precise Facial Expression Intensity Estimation](https://mlsanity.com/paper/2303.00180/) (2023)

## Built on by (most cited first, 30 of 110)

- [The 6th Affective Behavior Analysis in-the-wild (ABAW) Competition](https://mlsanity.com/paper/2402.19344/) (2024)
- [Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyond](https://mlsanity.com/paper/2401.01219/) (2024)
- [7th ABAW Competition: Multi-Task Learning and Compound Expression Recognition](https://mlsanity.com/paper/2407.03835/) (2024)
- [From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos](https://mlsanity.com/paper/2312.05447/) (2023)
- [Domain adaptation, Explainability & Fairness in AI for Medical Image Analysis: Diagnosis of COVID-19 based on 3-D Chest CT-scans](https://mlsanity.com/paper/2403.02192/) (2024)
- [Multi-modal Facial Affective Analysis based on Masked Autoencoder](https://mlsanity.com/paper/2303.10849/) (2023)
- [COVID-19 Computer-aided Diagnosis through AI-assisted CT Imaging Analysis: Deploying a Medical AI System](https://mlsanity.com/paper/2403.06242/) (2024)
- [Leveraging TCN and Transformer for effective visual-audio fusion in continuous emotion recognition](https://mlsanity.com/paper/2303.08356/) (2023)
- [BTDNet: a Multi-Modal Approach for Brain Tumor Radiogenomic Classification](https://mlsanity.com/paper/2310.03485/) (2023)
- [Affective Behaviour Analysis via Integrating Multi-Modal Knowledge](https://mlsanity.com/paper/2403.10825/) (2024)
- [Robust Light-Weight Facial Affective Behavior Recognition with CLIP](https://mlsanity.com/paper/2403.09915/) (2024)
- [Behaviour4All: in-the-wild Facial Behaviour Analysis Toolkit](https://mlsanity.com/paper/2409.17717/) (2024)
- [Recursive Joint Cross-Modal Attention for Multimodal Fusion in Dimensional Emotion Recognition](https://mlsanity.com/paper/2403.13659/) (2024)
- [HSEmotion Team at ABAW-8 Competition: Audiovisual Ambivalence/Hesitancy, Emotional Mimicry Intensity and Facial Expression Recognition](https://mlsanity.com/paper/2503.10399/) (2025)
- [Joint Multimodal Transformer for Emotion Recognition in the Wild](https://mlsanity.com/paper/2403.10488/) (2024)
- [MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild](https://mlsanity.com/paper/2404.09010/) (2024)
- [SAM2CLIP2SAM: Vision Language Model for Segmentation of 3D CT Scans for Covid-19 Detection](https://mlsanity.com/paper/2407.15728/) (2024)
- [DVD: A Comprehensive Dataset for Advancing Violence Detection in Real-World Scenarios](https://mlsanity.com/paper/2506.05372/) (2025)
- [ARBEx: Attentive Feature Extraction with Reliability Balancing for Robust Facial Expression Learning](https://mlsanity.com/paper/2305.01486/) (2023)
- [Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the Wild](https://mlsanity.com/paper/2407.12927/) (2024)
- [A transformer-based approach to video frame-level prediction in Affective Behaviour Analysis In-the-wild](https://mlsanity.com/paper/2303.09293/) (2023)
- [Multi-modal Facial Action Unit Detection with Large Pre-trained Models for the 5th Competition on Affective Behavior Analysis in-the-wild](https://mlsanity.com/paper/2303.10590/) (2023)
- [Exploring Facial Expression Recognition through Semi-Supervised Pretraining and Temporal Modeling](https://mlsanity.com/paper/2403.11942/) (2024)
- [Can Machine Learning Assist in Diagnosis of Primary Immune Thrombocytopenia? A feasibility study](https://mlsanity.com/paper/2405.20562/) (2024)
- [Affective Behavior Analysis using Task-adaptive and AU-assisted Graph Network](https://mlsanity.com/paper/2407.11663/) (2024)
- [Enhancing Facial Expression Recognition through Dual-Direction Attention Mixed Feature Networks: Application to 7th ABAW Challenge](https://mlsanity.com/paper/2407.12390/) (2024)
- [MAVEN: Multi-modal Attention for Valence-Arousal Emotion Network](https://mlsanity.com/paper/2503.12623/) (2025)
- [EmotiEffNet Facial Features in Uni-task Emotion Recognition in Video at ABAW-5 competition](https://mlsanity.com/paper/2303.09162/) (2023)
- [Spatio-Temporal AU Relational Graph Representation Learning For Facial Action Units Detection](https://mlsanity.com/paper/2303.10644/) (2023)
- [Affective Behaviour Analysis via Progressive Learning](https://mlsanity.com/paper/2407.16945/) (2024)

## Abstract (arXiv)

The fifth Affective Behavior Analysis in-the-wild (ABAW) Competition is part of the respective ABAW Workshop which will be held in conjunction with IEEE Computer Vision and Pattern Recognition Conference (CVPR), 2023. The 5th ABAW Competition is a continuation of the Competitions held at ECCV 2022, IEEE CVPR 2022, ICCV 2021, IEEE FG 2020 and CVPR 2017 Conferences, and is dedicated at automatically analyzing affect. For this year's Competition, we feature two corpora: i) an extended version of the Aff-Wild2 database and ii) the Hume-Reaction dataset. The former database is an audiovisual one of around 600 videos of around 3M frames and is annotated with respect to:a) two continuous affect dimensions -valence (how positive/negative a person is) and arousal (how active/passive a person is)-; b) basic expressions (e.g. happiness, sadness, neutral state); and c) atomic facial muscle actions (i.e., action units). The latter dataset is an audiovisual one in which reactions of individuals to emotional stimuli have been annotated with respect to seven emotional expression intensities. Thus the 5th ABAW Competition encompasses four Challenges: i) uni-task Valence-Arousal Estimation, ii) uni-task Expression Classification, iii) uni-task Action Unit Detection, and iv) Emotional Reaction Intensity Estimation. In this paper, we present these Challenges, along with their corpora, we outline the evaluation metrics, we present the baseline systems and illustrate their obtained performance.