mlsanityData to 7 Oct 2026

Boosting Continuous Emotion Recognition with Self-Pretraining using Masked Autoencoders, Temporal Convolutional Networks, and Transformers

arXiv 2403.11440 ↗First posted 18 Mar 2024cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

Human emotion recognition holds a pivotal role in facilitating seamless human-computer interaction. This paper delineates our methodology in tackling the Valence-Arousal (VA) Estimation Challenge, Expression (Expr) Classification Challenge, and Action Unit (AU) Detection Challenge within the ambit of the 6th Workshop and Competition on Affective Behavior Analysis in-the-wild (ABAW). Our study advocates a novel approach aimed at refining continuous emotion recognition. We achieve this by initially harnessing pre-training with Masked Autoencoders (MAE) on facial datasets, followed by fine-tuning on the aff-wild2 dataset annotated with expression (Expr) labels. The pre-trained model serves as an adept visual feature extractor, thereby enhancing the model's robustness. Furthermore, we bolster the performance of continuous emotion recognition by integrating Temporal Convolutional Network (TCN) modules and Transformer Encoder modules into our framework.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 17 of its 21 references that are in the map. First posted on arXiv 18 Mar 2024.

Builds on

17 of 21 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  3. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  4. 2016Temporal Convolutional Networks: A Unified Approach to Action Segmentationcited by 199
  5. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  6. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  7. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  8. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  9. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  10. 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
  11. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  12. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
5 more
  1. 2020Two-Stream Aural-Visual Affect Analysis in the Wildcited by 51
  2. 2021A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognitioncited by 24
  3. 2023Multi-modal Facial Affective Analysis based on Masked Autoencodercited by 23
  4. 2022Parallel Spatio-Temporal Attention-Based TCN for Multivariate Time Series Predictioncited by 12
  5. 2023Multi-modal Expression Recognition with Ensemble Methodcited by 3

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 3.

6 papers.

  1. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
  2. 2024Recursive Joint Cross-Modal Attention for Multimodal Fusion in Dimensional Emotion Recognitioncited by 15
  3. 2025MAVEN: Multi-modal Attention for Valence-Arousal Emotion Networkcited by 8
  4. 2025Mamba-VA: A Mamba-based Approach for Continuous Emotion Recognition in Valence-Arousal Spacecited by 6
  5. 2024AUD-TGN: Advancing Action Unit Detection with Temporal Convolution and GPT-2 in Wild Audiovisual Contextscited by 3
  6. 2025V-NAW: Video-based Noise-aware Adaptive Weighting for Facial Expression Recognitioncited by 1