mlsanityData to 7 Oct 2026

Vision Transformer for Action Units Detection

arXiv 2303.09917 ↗First posted 16 Mar 2023cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Facial Action Units detection (FAUs) represents a fine-grained classification problem that involves identifying different units on the human face, as defined by the Facial Action Coding System. In this paper, we present a simple yet efficient Vision Transformer-based approach for addressing the task of Action Units (AU) detection in the context of Affective Behavior Analysis in-the-wild (ABAW) competition. We employ the Video Vision Transformer(ViViT) Network to capture the temporal facial change in the video. Besides, to reduce massive size of the Vision Transformers model, we replace the ViViT feature extraction layers with the CNN backbone (Regnet). Our model outperform the baseline model of ABAW 2023 challenge, with a notable 14% difference in result. Furthermore, the achieved results are comparable to those of the top three teams in the previous ABAW 2022 challenge.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 19 of its 29 references that are in the map. First posted on arXiv 16 Mar 2023.

Builds on

19 of 29 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  3. 2021ViViT: A Video Vision Transformercited by 1,610
  4. 2020Designing Network Design Spacescited by 954
  5. 2017Focal Loss for Dense Object Detectioncited by 749
  6. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  7. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  8. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  9. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  10. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  11. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  12. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
7 more
  1. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  2. 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
  3. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  4. 2020JhatAA-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attentioncited by 62
  5. 2022Transformer-based Multimodal Information Fusion for Facial Expression Analysiscited by 49
  6. 2017EAC-Net: A Region-based Deep Enhancing and Cropping Approach for Facial Action Unit Detectioncited by 26
  7. 2022Multi-modal Multi-label Facial Action Unit Detection with Transformercited by 10

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2.

4 papers.

  1. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  2. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
  3. 2023Multi-modal Facial Affective Analysis based on Masked Autoencodercited by 23
  4. 2023Spatio-Temporal AU Relational Graph Representation Learning For Facial Action Units Detectioncited by 7