# Speech Emotion Recognition with Multiscale Area Attention and Data Augmentation

arXiv: https://arxiv.org/abs/2102.01813 · HTML: https://mlsanity.com/paper/2102.01813/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 24 references that are in the map.
- First posted on arXiv 3 Feb 2021.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Attention Driven Fusion for Multi-Modal Emotion Recognition](https://mlsanity.com/paper/2009.10991/) (2020)
- [Semi-Supervised Speech Emotion Recognition with Ladder Networks](https://mlsanity.com/paper/1905.02921/) (2019)
- [Area Attention](https://mlsanity.com/paper/1810.10126/) (2018)

## Built on by (most cited first, 12 of 12)

- [Dawn of the transformer era in speech emotion recognition: closing the valence gap](https://mlsanity.com/paper/2203.07378/) (2022)
- [Learning Speech Emotion Representations in the Quaternion Domain](https://mlsanity.com/paper/2204.02385/) (2022)
- [Designing and Evaluating Speech Emotion Recognition Systems: A reality check case study with IEMOCAP](https://mlsanity.com/paper/2304.00860/) (2023)
- [A New Amharic Speech Emotion Dataset and Classification Benchmark](https://mlsanity.com/paper/2201.02710/) (2022)
- [Speech Emotion Recognition with Global-Aware Fusion on Multi-scale Feature Representation](https://mlsanity.com/paper/2204.05571/) (2022)
- [Privacy against Real-Time Speech Emotion Detection via Acoustic Adversarial Evasion of Machine Learning](https://mlsanity.com/paper/2211.09273/) (2022)
- [Fusion with Hierarchical Graphs for Mulitmodal Emotion Recognition](https://mlsanity.com/paper/2109.07149/) (2021)
- [Speech and Text-Based Emotion Recognizer](https://mlsanity.com/paper/2312.11503/) (2023)
- [Searching for Effective Preprocessing Method and CNN-based Architecture with Efficient Channel Attention on Speech Emotion Recognition](https://mlsanity.com/paper/2409.04007/) (2024)
- [Less Stress, More Privacy: Stress Detection on Anonymized Speech of Air Traffic Controllers](https://mlsanity.com/paper/2507.08882/) (2025)
- [Transformer Redesign for Late Fusion of Audio-Text Features on Ultra-Low-Power Edge Hardware](https://mlsanity.com/paper/2510.18036/) (2025)
- [Multi-Loss Learning for Speech Emotion Recognition with Energy-Adaptive Mixup and Frame-Level Attention](https://mlsanity.com/paper/2512.04551/) (2025)

## Abstract (arXiv)

In Speech Emotion Recognition (SER), emotional characteristics often appear in diverse forms of energy patterns in spectrograms. Typical attention neural network classifiers of SER are usually optimized on a fixed attention granularity. In this paper, we apply multiscale area attention in a deep convolutional neural network to attend emotional characteristics with varied granularities and therefore the classifier can benefit from an ensemble of attentions with different scales. To deal with data sparsity, we conduct data augmentation with vocal tract length perturbation (VTLP) to improve the generalization capability of the classifier. Experiments are carried out on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) dataset. We achieved 79.34% weighted accuracy (WA) and 77.54% unweighted accuracy (UA), which, to the best of our knowledge, is the state of the art on this dataset.