# DST: Deformable Speech Transformer for Emotion Recognition

arXiv: https://arxiv.org/abs/2302.13729 · HTML: https://mlsanity.com/paper/2302.13729/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2023.

## Built on by (most cited first, 11 of 11)

- [emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation](https://mlsanity.com/paper/2312.15185/) (2023)
- [Vesper: A Compact and Effective Pretrained Model for Speech Emotion Recognition](https://mlsanity.com/paper/2307.10757/) (2023)
- [MFHCA: Enhancing Speech Emotion Recognition Via Multi-Spatial Fusion and Hierarchical Cooperative Attention](https://mlsanity.com/paper/2404.13509/) (2024)
- [A Change of Heart: Improving Speech Emotion Recognition through Speech-to-Text Modality Conversion](https://mlsanity.com/paper/2307.11584/) (2023)
- [Balancing Task-invariant Interaction and Task-specific Adaptation for Unified Image Fusion](https://mlsanity.com/paper/2504.05164/) (2025)
- [PTS-SNN: A Prompt-Tuned Temporal Shift Spiking Neural Networks for Efficient Speech Emotion Recognition](https://mlsanity.com/paper/2602.08240/) (2026)
- [Active Learning with Task Adaptation Pre-training for Speech Emotion Recognition](https://mlsanity.com/paper/2405.00307/) (2024)
- [EmoAugNet: A Signal-Augmented Hybrid CNN-LSTM Framework for Speech Emotion Recognition](https://mlsanity.com/paper/2508.06321/) (2025)
- [On the Contribution of Lexical Features to Speech Emotion Recognition](https://mlsanity.com/paper/2509.05634/) (2025)
- [Unrequited Emotions: Investigating the Gaps in Motivation and Practice in Speech Emotion Recognition Research](https://mlsanity.com/paper/2604.25776/) (2026)
- [Dual-Scale State-Space Modeling with Speaker-Wise Dynamic CRF for Speech Emotion Recognition in Conversation](https://mlsanity.com/paper/2608.22399/) (2026)

## Abstract (arXiv)

Enabled by multi-head self-attention, Transformer has exhibited remarkable results in speech emotion recognition (SER). Compared to the original full attention mechanism, window-based attention is more effective in learning fine-grained features while greatly reducing model redundancy. However, emotional cues are present in a multi-granularity manner such that the pre-defined fixed window can severely degrade the model flexibility. In addition, it is difficult to obtain the optimal window settings manually. In this paper, we propose a Deformable Speech Transformer, named DST, for SER task. DST determines the usage of window sizes conditioned on input speech via a light-weight decision network. Meanwhile, data-dependent offsets derived from acoustic features are utilized to adjust the positions of the attention windows, allowing DST to adaptively discover and attend to the valuable information embedded in the speech. Extensive experiments on IEMOCAP and MELD demonstrate the superiority of DST.