mlsanityData to 7 Oct 2026

Multi-modal Multi-label Facial Action Unit Detection with Transformer

arXiv 2203.13301 ↗First posted 24 Mar 2022cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Facial Action Coding System is an important approach of facial expression analysis.This paper describes our submission to the third Affective Behavior Analysis (ABAW) 2022 competition. We proposed a transfomer based model to detect facial action unit (FAU) in video. To be specific, we firstly trained a multi-modal model to extract both audio and visual feature. After that, we proposed a action units correlation module to learn relationships between each action unit labels and refine action unit detection result. Experimental results on validation dataset shows that our method achieves better performance than baseline model, which verifies that the effectiveness of proposed network.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 12 of its 16 references that are in the map. First posted on arXiv 24 Mar 2022.

Builds on

12 of 16 references.

References · 12

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  4. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  5. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  6. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  7. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  8. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  9. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
  10. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  11. 2020Two-Stream Aural-Visual Affect Analysis in the Wildcited by 51
  12. 2021A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognitioncited by 24

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 5; 2024: 4.

10 papers.

  1. 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
  2. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  3. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  4. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
  5. 2023Facial Affect Recognition based on Transformer Encoder and Audiovisual Fusion for the ABAW5 Challengecited by 6
  6. 2023Local Region Perception and Relationship Learning Combined with Feature Fusion for Facial Action Unit Detectioncited by 5
  7. 2023Vision Transformer for Action Units Detectioncited by 4
  8. 2023Spatial-temporal Transformer for Affective Behavior Analysiscited by 2
  9. 2024Facial Affect Recognition based on Multi Architecture Encoder and Feature Fusion for the ABAW7 Challengecited by 2
  10. 2024Compound Expression Recognition via Multi Model Ensemble for the ABAW7 Challengecited by 1