# Explainable speech emotion recognition through attentive pooling: insights from attention-based temporal localization

arXiv: https://arxiv.org/abs/2506.15754 · HTML: https://mlsanity.com/paper/2506.15754/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2025.

## Built on by (most cited first, 1 of 1)

- [Geometric Second-Order Feature Correlation Learning for Self-Supervised Speech Emotion Recognition](https://mlsanity.com/paper/2606.06550/) (2026)

## Abstract (arXiv)

State-of-the-art transformer models for Speech Emotion Recognition (SER) rely on temporal feature aggregation, yet advanced pooling methods remain underexplored. We systematically benchmark pooling strategies, including Multi-Query Multi-Head Attentive Statistics Pooling, which achieves a 3.5 percentage point macro F1 gain over average pooling. Attention analysis shows 15 percent of frames capture 80 percent of emotion cues, revealing a localized pattern of emotional information. Analysis of high-attention frames reveals that non-linguistic vocalizations and hyperarticulated phonemes are disproportionately prioritized during pooling, mirroring human perceptual strategies. Our findings position attentive pooling as both a performant SER mechanism and a biologically plausible tool for explainable emotion localization. On Interspeech 2025 Speech Emotion Recognition in Naturalistic Conditions Challenge, our approach obtained a macro F1 score of 0.3649.