mlsanityData to 14 Oct 2026

MAVEN: Multi-modal Attention for Valence-Arousal Emotion Network

arXiv 2503.12623 ↗First posted 16 Mar 2025cs.LG cs.AI cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Dynamic emotion recognition in the wild remains challenging due to the transient nature of emotional expressions and temporal misalignment of multi-modal cues. Traditional approaches predict valence and arousal and often overlook the inherent correlation between these two dimensions. The proposed Multi-modal Attention for Valence-Arousal Emotion Network (MAVEN) integrates visual, audio, and textual modalities through a bi-directional cross-modal attention mechanism. MAVEN uses modality-specific encoders to extract features from synchronized video frames, audio segments, and transcripts, predicting emotions in polar coordinates following Russell's circumplex model. The evaluation of the Aff-Wild2 dataset using MAVEN achieved a concordance correlation coefficient (CCC) of 0.3061, surpassing the ResNet-50 baseline model with a CCC of 0.22. The multistage architecture captures the subtle and transient nature of emotional expressions in conversational videos and improves emotion recognition in real-world situations. The code is available at: https://github.com/Vrushank-Ahire/MAVEN_8th_ABAW

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 47 of its 104 references that are in the map. First posted on arXiv 16 Mar 2025.

Builds on

47 of 104 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  6. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  7. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 511
  8. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  9. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  10. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  11. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  12. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
35 more
  1. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  2. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
  3. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  4. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  5. 2018Aff-Wild2: Extending the Aff-Wild Database for Affect Recognitioncited by 81
  6. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  7. 2020Multi-Head Attention: Collaborate Instead of Concatenatecited by 60
  8. 2024Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyondcited by 60
  9. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
  10. 2020Two-Stream Aural-Visual Affect Analysis in the Wildcited by 51
  11. 2019Exploiting multi-CNN features in CNN-RNN based Dimensional Emotion Recognition on the OMG in-the-wild Datasetcited by 39
  12. 2022A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognitioncited by 34
  13. 2025EmotiCrafter: Text-to-Emotional-Image Generation based on Valence-Arousal Modelcited by 26
  14. 2018A Multi-Task Learning & Generation Framework: Valence-Arousal, Action Units & Primary Expressionscited by 25
  15. 2018Interpretable Spatio-temporal Attention for Video Action Recognitioncited by 20
  16. 2024Affective Behaviour Analysis via Integrating Multi-Modal Knowledgecited by 18
  17. 2019MIMAMO Net: Integrating Micro- and Macro-motion for Video Emotion Recognitioncited by 17
  18. 2022Frame-level Prediction of Facial Expressions, Valence, Arousal and Action Units for Mobile Devicescited by 16
  19. 2022Transformer-Based Self-Supervised Learning for Emotion Recognitioncited by 16
  20. 2023Vesper: A Compact and Effective Pretrained Model for Speech Emotion Recognitioncited by 16
  21. 2022Multi-modal Emotion Estimation for in-the-wild Videoscited by 15
  22. 2024Recursive Joint Cross-Modal Attention for Multimodal Fusion in Dimensional Emotion Recognitioncited by 15
  23. 2025HSEmotion Team at ABAW-8 Competition: Audiovisual Ambivalence/Hesitancy, Emotional Mimicry Intensity and Facial Expression Recognitioncited by 15
  24. 2024Joint Multimodal Transformer for Emotion Recognition in the Wildcited by 14
  25. 2019Canonical Correlation Analysis (CCA) Based Multi-View Learning: An Overviewcited by 10
  26. 2024HSEmotion Team at the 6th ABAW Competition: Facial Expressions, Valence-Arousal and Emotion Intensity Predictioncited by 8
  27. 2021Technical Report for Valence-Arousal Estimation in ABAW2 Challengecited by 6
  28. 2024Boosting Continuous Emotion Recognition with Self-Pretraining using Masked Autoencoders, Temporal Convolutional Networks, and Transformerscited by 6
  29. 2025Emotion Recognition with CLIP and Sequential Learningcited by 6
  30. 2022Multi-Modal Learning for AU Detection Based on Multi-Head Fused Transformerscited by 5
  31. 2024Multimodal Fusion Method with Spatiotemporal Sequences and Relationship Learning for Valence-Arousal Estimationcited by 5
  32. 2024SUN Team's Contribution to ABAW 2024 Competition: Audio-visual Valence-Arousal Estimation and Expression Recognitioncited by 4
  33. 2024Emotion Recognition Using Transformers with Masked Learningcited by 4
  34. 2018Synchronous Prediction of Arousal and Valence Using LSTM Network for Affective Video Content Analysiscited by 1
  35. 2024Alignment Helps Make the Most of Multimodal Datacited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 8.

8 papers.

  1. 2026EmoSpace: Immersive Affective Image Generation Guided by Fine-Grained Emotion Prototypescited by 1
  2. 2026Stage-Adaptive Reliability Modeling for Continuous Valence-Arousal Estimationcited by 1
  3. 2026HSEmotion Team at ABAW-10 Competition: Facial Expression Recognition, Valence-Arousal Estimation, Action Unit Detection and Fine-Grained Violence Classificationcited by 1
  4. 2026LaScA: Language-Conditioned Scalable Modelling of Affective Dynamicscited by 0
  5. 2026Modeling Induced Pleasure through Cognitive Appraisal Prediction via Multimodal Fusioncited by 0
  6. 2026Quality-Aware Multimodal Fusion Reveals Implicit Identity in Valence-Arousal Featurescited by 0
  7. 2026Balancing Emotional Alignment and Semantic Consistency in Image Generation via Reinforcement Learning with Valence-Arousal Anchoringcited by 0
  8. 2026A Comprehensive Review of Multimodal Facial State Analysis: Tasks, Methods, and Resourcescited by 0