mlsanityData to 7 Oct 2026

Frame-level Prediction of Facial Expressions, Valence, Arousal and Action Units for Mobile Devices

arXiv 2203.13436 ↗First posted 25 Mar 2022cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

In this paper, we consider the problem of real-time video-based facial emotion analytics, namely, facial expression recognition, prediction of valence and arousal and detection of action unit points. We propose the novel frame-level emotion recognition algorithm by extracting facial features with the single EfficientNet model pre-trained on AffectNet. As a result, our approach may be implemented even for video analytics on mobile devices. Experimental results for the large scale Aff-Wild2 database from the third Affective Behavior Analysis in-the-wild (ABAW) Competition demonstrate that our simple model is significantly better when compared to the VggFace baseline. In particular, our method is characterized by 0.15-0.2 higher performance measures for validation sets in uni-task Expression Classification, Valence-Arousal Estimation and Expression Classification. Due to simplicity, our approach may be considered as a new baseline for all four sub-challenges.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 25 Mar 2022.

Builds on

25 of 36 references.

References · 25

  1. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  2. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  3. 2017VGGFace2: A dataset for recognising faces across pose and agecited by 911
  4. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 895
  5. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
  6. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  7. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  8. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  9. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  10. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  11. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  12. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
13 more
  1. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  2. 2019Frame attention networks for facial expression recognition in videoscited by 56
  3. 2022Transformer-based Multimodal Information Fusion for Facial Expression Analysiscited by 49
  4. 2021Facial expression and attributes recognition based on multi-task learning of lightweight neural networkscited by 40
  5. 2021A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognitioncited by 24
  6. 2021Iterative Distillation for Better Uncertainty Estimates in Multitask Emotion Recognitioncited by 20
  7. 2021Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competitioncited by 18
  8. 2018Efficient Facial Representations for Age, Gender and Identity Recognition in Organizing Photo Albums using Multi-output CNNcited by 10
  9. 2021Emotion Recognition with Incomplete Labels Using Modified Multi-task Learning Techniquecited by 10
  10. 2017Group-level Emotion Recognition using Transfer Learning from Face Identificationcited by 4
  11. 2022Challenges of Artificial Intelligence -- From Machine Learning and Computer Vision to Emotional Intelligencecited by 3
  12. 2019Preferences Prediction using a Gallery of Mobile Device based on Scene Recognition and Object Detectioncited by 1
  13. 2021Hotel Recognition via Latent Image Embeddingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 5; 2025: 2; 2026: 1.

16 papers.

Sort:

  1. 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
  2. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  3. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  4. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
  5. 2022A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognitioncited by 33
  6. 2024Facial Affective Behavior Analysis with Instruction Tuningcited by 26
  7. 2023Multi-modal Facial Affective Analysis based on Masked Autoencodercited by 23
  8. 2024Joint Multimodal Transformer for Emotion Recognition in the Wildcited by 13
  9. 2023FaceCLIPNeRF: Text-driven 3D Face Manipulation using Deformable Neural Radiance Fieldscited by 12
  10. 2022Affective Behavior Analysis using Action Unit Relation Graph and Multi-task Cross Attentioncited by 8
  11. 2023A transformer-based approach to video frame-level prediction in Affective Behaviour Analysis In-the-wildcited by 8
  12. 2024Enhancing Facial Expression Recognition through Dual-Direction Attention Mixed Feature Networks: Application to 7th ABAW Challengecited by 8
4 more
  1. 2025MAVEN: Multi-modal Attention for Valence-Arousal Emotion Networkcited by 8
  2. 2023Computer Vision Estimation of Emotion Reaction Intensity in the Wildcited by 2
  3. 2026A Two-Stage Dual-Modality Model for Facial Emotional Expression Recognitioncited by 1
  4. 2025Enhancing Facial Expression Recognition through Dual-Direction Attention Mixed Feature Networks and CLIP: Application to 8th ABAW Challengecited by 0