mlsanityData to 14 Oct 2026

Multi-modal Emotion Estimation for in-the-wild Videos

arXiv 2203.13032 ↗First posted 24 Mar 2022cs.CV eess.IVcited by 15

Not on the map. Why?

Abstract (arXiv)

In this paper, we briefly introduce our submission to the Valence-Arousal Estimation Challenge of the 3rd Affective Behavior Analysis in-the-wild (ABAW) competition. Our method utilizes the multi-modal information, i.e., the visual and audio information, and employs a temporal encoder to model the temporal context in the videos. Besides, a smooth processor is applied to get more reasonable predictions, and a model ensemble strategy is used to improve the performance of our proposed method. The experiment results show that our method achieves 65.55% ccc for valence and 70.88% ccc for arousal on the validation set of the Aff-Wild2 dataset, which prove the effectiveness of our proposed method.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 19 of its 32 references that are in the map. First posted on arXiv 24 Mar 2022.

Builds on

19 of 32 references.

References · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
  5. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 492
  6. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  7. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  8. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  9. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  10. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  11. 2020Partial FC: Training 10 Million Identities on a Single Machinecited by 163
  12. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
7 more
  1. 2016Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distributioncited by 158
  2. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
  3. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  4. 2014Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognitioncited by 111
  5. 2019Context-Aware Emotion Recognition Networkscited by 99
  6. 2020Improved Residual Networks for Image and Video Recognitioncited by 52
  7. 2017Temporal Multimodal Fusion for Video Emotion Classification in the Wildcited by 18

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 6; 2024: 5; 2025: 2.

15 papers.

Sort:

  1. 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
  2. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  3. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  4. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
  5. 2022A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognitioncited by 34
  6. 2025EmotiCrafter: Text-to-Emotional-Image Generation based on Valence-Arousal Modelcited by 26
  7. 2023Multi-modal Facial Affective Analysis based on Masked Autoencodercited by 23
  8. 2024Joint Multimodal Transformer for Emotion Recognition in the Wildcited by 14
  9. 2025MAVEN: Multi-modal Attention for Valence-Arousal Emotion Networkcited by 8
  10. 2023Facial Affect Recognition based on Transformer Encoder and Audiovisual Fusion for the ABAW5 Challengecited by 6
  11. 2024Multimodal Fusion Method with Spatiotemporal Sequences and Relationship Learning for Valence-Arousal Estimationcited by 5
  12. 2023Computer Vision Estimation of Emotion Reaction Intensity in the Wildcited by 3
3 more
  1. 2023Spatial-temporal Transformer for Affective Behavior Analysiscited by 2
  2. 2024Facial Affect Recognition based on Multi Architecture Encoder and Feature Fusion for the ABAW7 Challengecited by 2
  3. 2023NAC-TCN: Temporal Convolutional Networks with Causal Dilated Neighborhood Attention for Emotion Understandingcited by 0