mlsanityData to 14 Oct 2026

Spatio-Temporal AU Relational Graph Representation Learning For Facial Action Units Detection

arXiv 2303.10644 ↗First posted 19 Mar 2023cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

This paper presents our Facial Action Units (AUs) detection submission to the fifth Affective Behavior Analysis in-the-wild Competition (ABAW). Our approach consists of three main modules: (i) a pre-trained facial representation encoder which produce a strong facial representation from each input face image in the input sequence; (ii) an AU-specific feature generator that specifically learns a set of AU features from each facial representation; and (iii) a spatio-temporal graph learning module that constructs a spatio-temporal graph representation. This graph representation describes AUs contained in all frames and predicts the occurrence of each AU based on both the modeled spatial information within the corresponding face and the learned temporal dynamics among frames. The experimental results show that our approach outperformed the baseline and the spatio-temporal graph representation learning allows our model to generate the best results among all ablated systems. Our model ranks at the 4th place in the AU recognition track at the 5th ABAW Competition. Our code is publicly available at https://github.com/wzh125/ABAW-5.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 34 of its 60 references that are in the map. First posted on arXiv 19 Mar 2023.

Builds on

34 of 60 references.

References · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  6. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 492
  7. 2014Learning Face Representation from Scratchcited by 453
  8. 2017Gate-Variants of Gated Recurrent Unit (GRU) Neural Networkscited by 242
  9. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  10. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  11. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  12. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
22 more
  1. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  2. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  3. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
  4. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  5. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  6. 2022Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognitioncited by 82
  7. 2019Semantic Relationships Guided Representation Learning for Facial Action Unit Recognitioncited by 53
  8. 2022Transformer-based Multimodal Information Fusion for Facial Expression Analysiscited by 49
  9. 2018Facial Action Unit Detection Using Attention and Relation Learningcited by 46
  10. 2017Action Unit Detection with Region Adaptation, Multi-labeling Learning and Optimal Temporal Fusingcited by 28
  11. 2017EAC-Net: A Region-based Deep Enhancing and Cropping Approach for Facial Action Unit Detectioncited by 26
  12. 2021A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognitioncited by 24
  13. 2021Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competitioncited by 18
  14. 2022Facial Action Unit Detection and Intensity Estimation from Self-supervised Representationcited by 16
  15. 2022GRATIS: Deep Learning Graph Representation with Task-specific Topology and Multi-dimensional Edge Featurescited by 12
  16. 2022Affective Behavior Analysis using Action Unit Relation Graph and Multi-task Cross Attentioncited by 8
  17. 2023A transformer-based approach to video frame-level prediction in Affective Behaviour Analysis In-the-wildcited by 8
  18. 2023EmotiEffNet Facial Features in Uni-task Emotion Recognition in Video at ABAW-5 competitioncited by 7
  19. 2023Facial Affective Behavior Analysis Method for 5th ABAW Competitioncited by 6
  20. 2023Facial Affect Recognition based on Transformer Encoder and Audiovisual Fusion for the ABAW5 Challengecited by 6
  21. 2023Local Region Perception and Relationship Learning Combined with Feature Fusion for Facial Action Unit Detectioncited by 5
  22. 2023Vision Transformer for Action Units Detectioncited by 4

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 6.

7 papers.

  1. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  2. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
  3. 2023Multi-modal Facial Affective Analysis based on Masked Autoencodercited by 23
  4. 2024Multi-scale Dynamic and Hierarchical Relationship Modeling for Facial Action Units Recognitioncited by 11
  5. 2024Emotion Recognition Using Transformers with Masked Learningcited by 4
  6. 2024AUD-TGN: Advancing Action Unit Detection with Temporal Convolution and GPT-2 in Wild Audiovisual Contextscited by 3
  7. 2024AU-vMAE: Knowledge-Guide Action Units Detection via Video Masked Autoencodercited by 2