Audio-Text Cross-Attention with Psycholinguistic Support Features for Ambivalence/Hesitancy Recognition
Not on the map. Why?
Abstract (arXiv)
We present a frame-independent audio-text system for the 3rd Ambivalence/Hesitancy Video Recognition Challenge at the 11th Affective & Behavior Analysis in-the-Wild (ABAW) Workshop. Videos are divided into overlapping 5-s windows aligned with transcript timestamps. Each window combines prosodic audio descriptors, emotion-oriented RoBERTa embeddings, and 74 psycholinguistic features representing uncertainty, hedging, and attitudinal conflict. Temporal cross-attention fuses audio and text, while the support features condition gated Multiple Instance Learning (MIL) pooling. A five-seed ensemble achieves an average precision of 0.875 and a macro-F1 of 0.722 on the 525-video labeled public-test split. Notably, our submission ranked third overall on the official challenge leaderboard, with a macro-F1 of 0.7455. Source code is available at https://github.com/Liga-de-IA-PUCPR/abaw-11-ah-challenge/.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 43 references that are in the map. First posted on arXiv 15 Jul 2026.
Builds on
11 of 43 references.
References · 11
- 2017Attention Is All You Needcited by 59,932
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
- 2018Attention-based Deep Multiple Instance Learningcited by 1,077
- 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
- 2023Proposal-Based Multiple Instance Learning for Weakly-Supervised Temporal Action Localizationcited by 19
- 2025BAH Dataset for Ambivalence/Hesitancy Recognition in Videos for Digital Behavioural Changecited by 14
- 2026From Affect to Complex Behavior: Advancing Multimodal Human-Centered AI at the 10th ABAW Workshop & Competitioncited by 12
- 2026Conflict-Aware Multimodal Fusion for Ambivalence and Hesitancy Recognitioncited by 11
- 2026Multimodal Ambivalence/Hesitancy Recognition in Videos for Personalized Digital Health Interventionscited by 3
- 2026Simple Features and Honest Calibration for Ambivalence and Hesitancy Recognition in Videocited by 1
- 2026HSEmotion Team at the 11th ABAW Challenge: Multi-Task Learning and Ambivalence/Hesitancy Video Recognitioncited by 1
Built on by
Not cited yet.