mlsanityData to 14 Oct 2026

Multimodal Emotion Regression with Multi-Objective Optimization and VAD-Aware Audio Modeling for the 10th ABAW EMI Track

arXiv 2603.13760 ↗First posted 14 Mar 2026cs.AI cs.SDcited by 2

Not on the map. Why?

Abstract (arXiv)

We participated in the 10th ABAW Challenge, focusing on the Emotional Mimicry Intensity (EMI) Estimation track on the Hume-Vidmimic2 dataset. This task aims to predict six continuous emotion dimensions: Admiration, Amusement, Determination, Empathic Pain, Excitement, and Joy. Through systematic multimodal exploration of pretrained high-level features, we found that, under our pretrained feature setting, direct feature concatenation outperformed the more complex fusion strategies we tested. This empirical finding motivated us to design a systematic approach built upon three core principles: (i) preserving modality-specific attributes through feature-level concatenation; (ii) improving training stability and metric alignment via multi-objective optimization; and (iii) enriching acoustic representations with a VAD-inspired latent prior. Our final framework integrates concatenation-based multimodal fusion, a shared six-dimensional regression head, multi-objective optimization with MSE, Pearson-correlation, and auxiliary branch supervision, EMA for parameter stabilization, and a VAD-inspired latent prior for the acoustic branch. On the official validation set, the proposed scheme achieved our best mean Pearson Correlation Coefficient of 0.478567.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 17 of its 22 references that are in the map. First posted on arXiv 14 Mar 2026.

Builds on

17 of 22 references.

References · 17

  1. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  2. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  3. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  4. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  5. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  6. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  7. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
  8. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  9. 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
  10. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  11. 2024Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyondcited by 60
  12. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
5 more
  1. 2024Behaviour4All: in-the-wild Facial Behaviour Analysis Toolkitcited by 16
  2. 2025HSEmotion Team at ABAW-8 Competition: Audiovisual Ambivalence/Hesitancy, Emotional Mimicry Intensity and Facial Expression Recognitioncited by 15
  3. 2025DVD: A Comprehensive Dataset for Advancing Violence Detection in Real-World Scenarioscited by 13
  4. 2024Efficient Feature Extraction and Late Fusion Strategy for Audiovisual Emotional Mimicry Intensity Estimationcited by 6
  5. 2024Unimodal Multi-Task Fusion for Emotional Mimicry Intensity Predictioncited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026From Affect to Complex Behavior: Advancing Multimodal Human-Centered AI at the 10th ABAW Workshop & Competitioncited by 12
  2. 2026Two-Stage Multimodal Framework for Emotion Mimicry Intensity Predictioncited by 0