mlsanityData to 7 Oct 2026

Mutilmodal Feature Extraction and Attention-based Fusion for Emotion Estimation in Videos

arXiv 2303.10421 ↗First posted 18 Mar 2023cs.CV cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

The continuous improvement of human-computer interaction technology makes it possible to compute emotions. In this paper, we introduce our submission to the CVPR 2023 Competition on Affective Behavior Analysis in-the-wild (ABAW). Sentiment analysis in human-computer interaction should, as far as possible Start with multiple dimensions, fill in the single imperfect emotion channel, and finally determine the emotion tendency by fitting multiple results. Therefore, We exploited multimodal features extracted from video of different lengths from the competition dataset, including audio, pose and images. Well-informed emotion representations drive us to propose a Attention-based multimodal framework for emotion estimation. Our system achieves the performance of 0.361 on the validation dataset. The code is available at [https://github.com/xkwangcn/ABAW-5th-RT-IAI].

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 19 of its 29 references that are in the map. First posted on arXiv 18 Mar 2023.

Builds on

19 of 29 references.

References · 19

  1. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,199
  2. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  3. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  4. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  5. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  6. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  7. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  8. 2020Suppressing Uncertainties for Large-Scale Facial Expression Recognitioncited by 157
  9. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
  10. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  11. 2019Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognitioncited by 127
  12. 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
7 more
  1. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  2. 2019STEP: Spatial Temporal Graph Convolutional Networks for Emotion Perception from Gaitscited by 36
  3. 2016FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognitioncited by 25
  4. 2020ProxEmo: Gait-based Emotion Learning and Multi-view Proxemic Fusion for Socially-Aware Robot Navigationcited by 17
  5. 2020Occlusion-Adaptive Deep Network for Robust Facial Expression Recognitioncited by 17
  6. 2020BReG-NeXt: Facial Affect Computing Using Adaptive Residual Networks With Bounded Gradientcited by 13
  7. 2019Identifying Emotions from Walking using Affective and Deep Featurescited by 12

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  2. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53