Technical Approach for the EMI Challenge in the 8th Affective Behavior Analysis in-the-Wild Competition
Not on the map. Why?
Abstract (arXiv)
Emotional Mimicry Intensity (EMI) estimation plays a pivotal role in understanding human social behavior and advancing human-computer interaction. The core challenges lie in dynamic correlation modeling and robust fusion of multimodal temporal signals. To address the limitations of existing methods--insufficient exploitation of cross-modal synergies, sensitivity to noise, and constrained fine-grained alignment capabilities--this paper proposes a dual-stage cross-modal alignment framework. Stage 1 develops vision-text and audio-text contrastive learning networks based on a CLIP architecture, achieving preliminary feature-space alignment through modality-decoupled pre-training. Stage 2 introduces a temporal-aware dynamic fusion module integrating Temporal Convolutional Networks (TCN) and gated bidirectional LSTM to capture macro-evolution patterns of facial expressions and local dynamics of acoustic features, respectively. A novel quality-guided fusion strategy further enables differentiable weight allocation for modality compensation under occlusion and noise. Experiments on the Hume-Vidmimic2 dataset demonstrate superior performance with an average Pearson correlation coefficient of 0.51 across six emotion dimensions on the validate set. Remarkably, our method achieved 0.68 on the test set, securing runner-up in the EMI Challenge Track of the 8th ABAW (Affective Behavior Analysis in the Wild) Competition, offering a novel pathway for fine-grained emotion analysis in open environments.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 25 of its 35 references that are in the map. First posted on arXiv 13 Mar 2025.
Builds on
25 of 35 references.
References · 25
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
- 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
- 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
- 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 433
- 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 265
- 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
- 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
13 more
- 2016Temporal Convolutional Networks: A Unified Approach to Action Segmentationcited by 199
- 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
- 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
- 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
- 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
- 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
- 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
- 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
- 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
- 2018Aff-Wild2: Extending the Aff-Wild Database for Affect Recognitioncited by 81
- 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
- 2024Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyondcited by 60
- 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
Built on by
Citing papers per year: 2026: 2.
2 papers.