mlsanityData to 7 Oct 2026

Audio-Text Cross-Attention with Psycholinguistic Support Features for Ambivalence/Hesitancy Recognition

arXiv 2607.13345 ↗First posted 15 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a frame-independent audio-text system for the 3rd Ambivalence/Hesitancy Video Recognition Challenge at the 11th Affective & Behavior Analysis in-the-Wild (ABAW) Workshop. Videos are divided into overlapping 5-s windows aligned with transcript timestamps. Each window combines prosodic audio descriptors, emotion-oriented RoBERTa embeddings, and 74 psycholinguistic features representing uncertainty, hedging, and attitudinal conflict. Temporal cross-attention fuses audio and text, while the support features condition gated Multiple Instance Learning (MIL) pooling. A five-seed ensemble achieves an average precision of 0.875 and a macro-F1 of 0.722 on the 525-video labeled public-test split. Notably, our submission ranked third overall on the official challenge leaderboard, with a macro-F1 of 0.7455. Source code is available at https://github.com/Liga-de-IA-PUCPR/abaw-11-ah-challenge/.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 43 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

11 of 43 references.

References · 11

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2018Attention-based Deep Multiple Instance Learningcited by 1,077
  4. 2020TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classificationcited by 414
  5. 2023Proposal-Based Multiple Instance Learning for Weakly-Supervised Temporal Action Localizationcited by 19
  6. 2025BAH Dataset for Ambivalence/Hesitancy Recognition in Videos for Digital Behavioural Changecited by 14
  7. 2026From Affect to Complex Behavior: Advancing Multimodal Human-Centered AI at the 10th ABAW Workshop & Competitioncited by 12
  8. 2026Conflict-Aware Multimodal Fusion for Ambivalence and Hesitancy Recognitioncited by 11
  9. 2026Multimodal Ambivalence/Hesitancy Recognition in Videos for Personalized Digital Health Interventionscited by 3
  10. 2026Simple Features and Honest Calibration for Ambivalence and Hesitancy Recognition in Videocited by 1
  11. 2026HSEmotion Team at the 11th ABAW Challenge: Multi-Task Learning and Ambivalence/Hesitancy Video Recognitioncited by 1

Built on by

Not cited yet.