# Multimodal Continuous Emotion Recognition: A Technical Report for ABAW5

arXiv: https://arxiv.org/abs/2303.10335 · HTML: https://mlsanity.com/paper/2303.10335/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2023.

## Built on by (most cited first, 14 of 14)

- [The 6th Affective Behavior Analysis in-the-wild (ABAW) Competition](https://mlsanity.com/paper/2402.19344/) (2024)
- [7th ABAW Competition: Multi-Task Learning and Compound Expression Recognition](https://mlsanity.com/paper/2407.03835/) (2024)
- [Multi-modal Facial Affective Analysis based on Masked Autoencoder](https://mlsanity.com/paper/2303.10849/) (2023)
- [Recursive Joint Cross-Modal Attention for Multimodal Fusion in Dimensional Emotion Recognition](https://mlsanity.com/paper/2403.13659/) (2024)
- [BAH Dataset for Ambivalence/Hesitancy Recognition in Videos for Digital Behavioural Change](https://mlsanity.com/paper/2505.19328/) (2025)
- [HSEmotion Team at the 6th ABAW Competition: Facial Expressions, Valence-Arousal and Emotion Intensity Prediction](https://mlsanity.com/paper/2403.11590/) (2024)
- [Trends in Machine Learning and Electroencephalogram (EEG): A Review for Undergraduate Researchers](https://mlsanity.com/paper/2307.02819/) (2023)
- [Multimodal Fusion Method with Spatiotemporal Sequences and Relationship Learning for Valence-Arousal Estimation](https://mlsanity.com/paper/2403.12425/) (2024)
- [Emotion Recognition Using Transformers with Masked Learning](https://mlsanity.com/paper/2403.13731/) (2024)
- [Multimodal Ambivalence/Hesitancy Recognition in Videos for Personalized Digital Health Interventions](https://mlsanity.com/paper/2604.11730/) (2026)
- [Grounding Emotion Recognition with Visual Prototypes: VEGA -- Revisiting CLIP in MERC](https://mlsanity.com/paper/2508.06564/) (2025)
- [NAC-TCN: Temporal Convolutional Networks with Causal Dilated Neighborhood Attention for Emotion Understanding](https://mlsanity.com/paper/2312.07507/) (2023)
- [United we stand, Divided we fall: Handling Weak Complementary Relationships for Audio-Visual Emotion Recognition in Valence-Arousal Space](https://mlsanity.com/paper/2503.12261/) (2025)
- [Modality Discrepancy Transformer for Ambivalence and Hesitancy Recognition](https://mlsanity.com/paper/2609.19148/) (2026)

## Abstract (arXiv)

We used two multimodal models for continuous valence-arousal recognition using visual, audio, and linguistic information. The first model is the same as we used in ABAW2 and ABAW3, which employs the leader-follower attention. The second model has the same architecture for spatial and temporal encoding. As for the fusion block, it employs a compact and straightforward channel attention, borrowed from the End2You toolkit. Unlike our previous attempts that use Vggish feature directly as the audio feature, this time we feed the pre-trained VGG model using logmel-spectrogram and finetune it during the training. To make full use of the data and alleviate over-fitting, cross-validation is carried out. The code is available at https://github.com/sucv/ABAW3.