Team RAS in 10th ABAW Competition: Multimodal Valence and Arousal Estimation Approach
Not on the map. Why?
Abstract (arXiv)
Continuous emotion recognition in terms of valence and arousal under in-the-wild (ITW) conditions remains a challenging problem due to large variations in appearance, head pose, illumination, occlusions, and subject-specific patterns of affective expression. We present a multimodal method for valence-arousal estimation ITW. Our method combines three complementary modalities: face, behavior, and audio. The face modality relies on GRADA-based frame-level embeddings and Transformer-based temporal regression. We use Qwen3-VL-4B-Instruct to extract behavior-relevant information from video segments, while Mamba is used to model temporal dynamics across segments. The audio modality relies on WavLM-Large with attention-statistics pooling and includes a cross-modal filtering stage to reduce the influence of unreliable or non-speech segments. To fuse modalities, we explore two fusion strategies: a Directed Cross-Modal Mixture-of-Experts Fusion Strategy that learns interactions between modalities with adaptive weighting, and a Reliability-Aware Audio-Visual Fusion Strategy that combines visual features at the frame-level while using audio as complementary context. The results are reported on the Aff-Wild2 dataset following the 10th Affective Behavior Analysis in-the-Wild (ABAW) challenge protocol. Experiments demonstrate that the proposed multimodal fusion strategy achieves a Concordance Correlation Coefficient (CCC) of 0.658 on the Aff-Wild2 development set.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 24 of its 44 references that are in the map. First posted on arXiv 13 Mar 2026.
Builds on
24 of 44 references.
References · 24
- 2017Attention Is All You Needcited by 60,361
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 648
- 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 642
- 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
- 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
- 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
- 2018MobileFaceNets: Efficient CNNs for Accurate Real-Time Face Verification on Mobile Devicescited by 182
- 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
- 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
12 more
- 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
- 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
- 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
- 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
- 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
- 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
- 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
- 2024Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyondcited by 60
- 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
- 2024Behaviour4All: in-the-wild Facial Behaviour Analysis Toolkitcited by 16
- 2025DVD: A Comprehensive Dataset for Advancing Violence Detection in Real-World Scenarioscited by 13
- 2025Emotion Recognition with CLIP and Sequential Learningcited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Distance-aware Soft Prompt Guidance for Multimodal Valence-Arousal Estimation
- 2026Team LEYA in 10th ABAW Competition: Multimodal Ambivalence/Hesitancy Recognition Approach
- 2026Emotion as a Distribution: Joint Valence-Arousal Probability Learning for Speaker-Independent Multimodal Emotion Recognition
- 2026Foundation Model Embeddings Meet Blended Emotions: A Multimodal Fusion Approach for the BLEMORE Challenge
- 2025Omni-Emotion: Extending Video MLLM with Detailed Face and Audio Modeling for Multimodal Emotion Analysis
- 2026AffectFuse: Cross-Task Feature Fusion with Temporal Modeling for Multi-Task Affective Behavior Analysis
- 2025Mamba-VA: A Mamba-based Approach for Continuous Emotion Recognition in Valence-Arousal Space
- 2026Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding