# Mingling or Misalignment? Temporal Shift for Speech Emotion Recognition with Pre-trained Representations

arXiv: https://arxiv.org/abs/2302.13277 · HTML: https://mlsanity.com/paper/2302.13277/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2023.

## Built on by (most cited first, 5 of 5)

- [Vesper: A Compact and Effective Pretrained Model for Speech Emotion Recognition](https://mlsanity.com/paper/2307.10757/) (2023)
- [Two-stage Framework for Robust Speech Emotion Recognition Using Target Speaker Extraction in Human Speech Noise Conditions](https://mlsanity.com/paper/2409.19585/) (2024)
- [DSNet: Disentangled Siamese Network with Neutral Calibration for Speech Emotion Recognition](https://mlsanity.com/paper/2312.15593/) (2023)
- [PTS-SNN: A Prompt-Tuned Temporal Shift Spiking Neural Networks for Efficient Speech Emotion Recognition](https://mlsanity.com/paper/2602.08240/) (2026)
- [Emotion Loss Attacking: Adversarial Attack Perception for Skeleton based on Multi-dimensional Features](https://mlsanity.com/paper/2406.19815/) (2024)

## Abstract (arXiv)

Fueled by recent advances of self-supervised models, pre-trained speech representations proved effective for the downstream speech emotion recognition (SER) task. Most prior works mainly focus on exploiting pre-trained representations and just adopt a linear head on top of the pre-trained model, neglecting the design of the downstream network. In this paper, we propose a temporal shift module to mingle channel-wise information without introducing any parameter or FLOP. With the temporal shift module, three designed baseline building blocks evolve into corresponding shift variants, i.e. ShiftCNN, ShiftLSTM, and Shiftformer. Moreover, to balance the trade-off between mingling and misalignment, we propose two technical strategies, placement of shift and proportion of shift. The family of temporal shift models all outperforms the state-of-the-art methods on the benchmark IEMOCAP dataset under both finetuning and feature extraction settings. Our code is available at https://github.com/ECNU-Cross-Innovation-Lab/ShiftSER.