mlsanityData to 7 Oct 2026

SUN Team's Contribution to ABAW 2024 Competition: Audio-visual Valence-Arousal Estimation and Expression Recognition

arXiv 2403.12609 ↗First posted 19 Mar 2024cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

As emotions play a central role in human communication, automatic emotion recognition has attracted increasing attention in the last two decades. While multimodal systems enjoy high performances on lab-controlled data, they are still far from providing ecological validity on non-lab-controlled, namely 'in-the-wild' data. This work investigates audiovisual deep learning approaches for emotion recognition in-the-wild problem. We particularly explore the effectiveness of architectures based on fine-tuned Convolutional Neural Networks (CNN) and Public Dimensional Emotion Model (PDEM), for video and audio modality, respectively. We compare alternative temporal modeling and fusion strategies using the embeddings from these multi-stage trained modality-specific Deep Neural Networks (DNN). We report results on the AffWild2 dataset under Affective Behavior Analysis in-the-Wild 2024 (ABAW'24) challenge protocol.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 25 of its 49 references that are in the map. First posted on arXiv 19 Mar 2024.

Builds on

25 of 49 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  7. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 572
  8. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
  9. 2013Challenges in Representation Learning: A report on three machine learning contestscited by 301
  10. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  11. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  12. 2020Does label smoothing mitigate label noise?cited by 204
13 more
  1. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  2. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  3. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  4. 2019RetinaFace: Single-stage Dense Face Localisation in the Wildcited by 164
  5. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  6. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
  7. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  8. 2022Dawn of the transformer era in speech emotion recognition: closing the valence gapcited by 155
  9. 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
  10. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  11. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  12. 2019SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wildcited by 62
  13. 2016From Facial Expression Recognition to Interpersonal Relation Predictioncited by 53

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
  2. 2024Recursive Joint Cross-Modal Attention for Multimodal Fusion in Dimensional Emotion Recognitioncited by 15
  3. 2025MAVEN: Multi-modal Attention for Valence-Arousal Emotion Networkcited by 8
  4. 2026A Two-Stage Dual-Modality Model for Facial Emotional Expression Recognitioncited by 1