mlsanityData to 7 Oct 2026

Continuous Emotion Recognition using Visual-audio-linguistic information: A Technical Report for ABAW3

arXiv 2203.13031 ↗First posted 24 Mar 2022cs.MM cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

We propose a cross-modal co-attention model for continuous emotion recognition using visual-audio-linguistic information. The model consists of four blocks. The visual, audio, and linguistic blocks are used to learn the spatial-temporal features of the multi-modal input. A co-attention block is designed to fuse the learned features with the multi-head co-attention mechanism. The visual encoding from the visual block is concatenated with the attention feature to emphasize the visual information. To make full use of the data and alleviate over-fitting, cross-validation is carried out on the training and validation set. The concordance correlation coefficient (CCC) centering is used to merge the results from each fold. The achieved CCC on the test set is 0.520 for valence and 0.602 for arousal, which significantly outperforms the baseline method with the corresponding CCC of 0.180 and 0.170 for valence and arousal, respectively. The code is available at https://github.com/sucv/ABAW3.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 17 of its 32 references that are in the map. First posted on arXiv 24 Mar 2022.

Builds on

17 of 32 references.

References · 17

  1. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,447
  2. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  3. 2016MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognitioncited by 621
  4. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  5. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  6. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  7. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  8. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  9. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  10. 2016Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distributioncited by 158
  11. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
  12. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
5 more
  1. 2019AVEC 2019 Workshop and Challenge: State-of-Mind, Detecting Depression with AI, and Cross-Cultural Affect Recognitioncited by 87
  2. 2016AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and Challengecited by 68
  3. 2022Transformer-based Multimodal Information Fusion for Facial Expression Analysiscited by 49
  4. 2021Continuous Emotion Recognition with Audio-visual Leader-follower Attentive Fusioncited by 15
  5. 2021MuSe-Toolbox: The Multimodal Sentiment Analysis Continuous Annotation Fusion and Discrete Class Transformation Toolboxcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 8; 2024: 5; 2025: 1; 2026: 3.

20 papers.

Sort:

  1. 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
  2. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  3. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  4. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
  5. 2022A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognitioncited by 33
  6. 2023Multi-modal Facial Affective Analysis based on Masked Autoencodercited by 23
  7. 2024Joint Multimodal Transformer for Emotion Recognition in the Wildcited by 13
  8. 2022Multi-Task Learning Framework for Emotion Recognition in-the-wildcited by 10
  9. 2024Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the Wildcited by 9
  10. 2023EmotiEffNet Facial Features in Uni-task Emotion Recognition in Video at ABAW-5 competitioncited by 7
  11. 2023Facial Affect Recognition based on Transformer Encoder and Audiovisual Fusion for the ABAW5 Challengecited by 6
  12. 2025Semantic Matters: Multimodal Features for Affective Analysiscited by 5
8 more
  1. 2026Solution for 10th Competition on Ambivalence/Hesitancy (AH) Video Recognition Challenge using Divergence-Based Multimodal Fusioncited by 4
  2. 2023Spatial-temporal Transformer for Affective Behavior Analysiscited by 2
  3. 2023Computer Vision Estimation of Emotion Reaction Intensity in the Wildcited by 2
  4. 2023Accommodating Missing Modalities in Time-Continuous Multimodal Emotion Recognitioncited by 2
  5. 2024Facial Affect Recognition based on Multi Architecture Encoder and Feature Fusion for the ABAW7 Challengecited by 2
  6. 2026Multimodal Emotion Recognition via Bi-directional Cross-Attention and Temporal Modelingcited by 1
  7. 2023NAC-TCN: Temporal Convolutional Networks with Causal Dilated Neighborhood Attention for Emotion Understandingcited by 0
  8. 2026Emotion-Aware Classroom Quality Assessment Leveraging IoT-Based Real-Time Student Monitoringcited by 0