mlsanityData to 7 Oct 2026

Solution for 10th Competition on Ambivalence/Hesitancy (AH) Video Recognition Challenge using Divergence-Based Multimodal Fusion

arXiv 2603.16939 ↗First posted 15 Mar 2026cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

We address the Ambivalence/Hesitancy (A/H) Video Recognition Challenge at the 10th ABAW Competition (CVPR 2026). We propose a divergence-based multimodal fusion that explicitly measures cross-modal conflict between visual, audio, and textual channels. Visual features are encoded as Action Units (AUs) extracted via Py-Feat, audio via Wav2Vec 2.0, and text via BERT. Each modality is processed by a BiLSTM with attention pooling and projected into a shared embedding space. The fusion module computes pairwise absolute differences between modality embeddings, directly capturing the incongruence that characterizes A/H. On the BAH dataset, our approach achieves a Macro F1 of 0.6808 on the validation test set, outperforming the challenge baseline of 0.2827. Statistical analysis across 1{,}132 videos confirms that temporal variability of AUs is the dominant visual discriminator of A/H.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 24 of its 32 references that are in the map. First posted on arXiv 15 Mar 2026.

Builds on

24 of 32 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  6. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  7. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 572
  8. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
  9. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
  10. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  11. 2018Deep Facial Expression Recognition: A Surveycited by 269
  12. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
12 more
  1. 2022Dawn of the transformer era in speech emotion recognition: closing the valence gapcited by 155
  2. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  3. 2017End-to-End Multimodal Emotion Recognition using Deep Neural Networkscited by 69
  4. 2020JhatAA-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attentioncited by 62
  5. 2021Py-Feat: Python Facial Expression Analysis Toolboxcited by 33
  6. 2022Continuous Emotion Recognition using Visual-audio-linguistic information: A Technical Report for ABAW3cited by 20
  7. 2025HSEmotion Team at ABAW-8 Competition: Audiovisual Ambivalence/Hesitancy, Emotional Mimicry Intensity and Facial Expression Recognitioncited by 15
  8. 2025BAH Dataset for Ambivalence/Hesitancy Recognition in Videos for Digital Behavioural Changecited by 14
  9. 2024Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the Wildcited by 9
  10. 2024Efficient Feature Extraction and Late Fusion Strategy for Audiovisual Emotional Mimicry Intensity Estimationcited by 6
  11. 2024Unimodal Multi-Task Fusion for Emotional Mimicry Intensity Predictioncited by 6
  12. 2025Semantic Matters: Multimodal Features for Affective Analysiscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.