mlsanityData to 7 Oct 2026

Solution to the 10th ABAW Expression Recognition Challenge: A Robust Multimodal Framework with Safe Cross-Attention and Modality Dropout

arXiv 2603.08034 ↗First posted 9 Mar 2026cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Emotion recognition in real-world environments is hindered by partial occlusions, missing modalities, and severe class imbalance. To address these issues, particularly for the Affective Behavior Analysis in-the-wild (ABAW) Expression challenge, we propose a multimodal framework that dynamically fuses visual and audio representations. Our approach uses a dual-branch Transformer architecture featuring a safe cross-attention mechanism and a modality dropout strategy. This design allows the network to rely on audio-based predictions when visual cues are absent. To mitigate the long-tail distribution of the Aff-Wild2 dataset, we apply focal loss optimization, combined with a sliding-window soft voting strategy to capture dynamic emotional transitions and reduce frame-level classification jitter. Experiments demonstrate that our framework effectively handles missing modalities and complex spatiotemporal dependencies, achieving an accuracy of 60.79% and an F1-score of 0.5029 on the Aff-Wild2 validation set.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 45 references that are in the map. First posted on arXiv 9 Mar 2026.

Builds on

29 of 45 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  6. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,239
  7. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
  8. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
  9. 2017Gated Multimodal Units for Information Fusioncited by 244
  10. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  11. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  12. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
17 more
  1. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  2. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  3. 2016Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distributioncited by 158
  4. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
  5. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  6. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  7. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  8. 2024Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyondcited by 60
  9. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
  10. 2024Affective Behaviour Analysis via Integrating Multi-Modal Knowledgecited by 18
  11. 2024Robust Light-Weight Facial Affective Behavior Recognition with CLIPcited by 16
  12. 2024Behaviour4All: in-the-wild Facial Behaviour Analysis Toolkitcited by 16
  13. 2025HSEmotion Team at ABAW-8 Competition: Audiovisual Ambivalence/Hesitancy, Emotional Mimicry Intensity and Facial Expression Recognitioncited by 15
  14. 2022Empathetic Conversational Systems: A Review of Current Advances, Gaps, and Opportunitiescited by 14
  15. 2025DVD: A Comprehensive Dataset for Advancing Violence Detection in Real-World Scenarioscited by 13
  16. 2024Exploring Facial Expression Recognition through Semi-Supervised Pretraining and Temporal Modelingcited by 8
  17. 2025Emotion Recognition with CLIP and Sequential Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.