# DSNet: Disentangled Siamese Network with Neutral Calibration for Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2312.15593 · HTML: https://mlsanity.com/paper/2312.15593/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 25 Dec 2023.

## Builds on (most cited first, 15 of 15)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Domain Separation Networks](https://mlsanity.com/paper/1608.06019/) (2016)
- [Dawn of the transformer era in speech emotion recognition: closing the valence gap](https://mlsanity.com/paper/2203.07378/) (2022)
- [Disentangled Representation Learning](https://mlsanity.com/paper/2211.11695/) (2022)
- [SigNet: Convolutional Siamese Network for Writer Independent Offline Signature Verification](https://mlsanity.com/paper/1707.02131/) (2017)
- [Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversion](https://mlsanity.com/paper/2203.16705/) (2022)
- [Domain Adversarial for Acoustic Emotion Recognition](https://mlsanity.com/paper/1804.07690/) (2018)
- [Temporal Modeling Matters: A Novel Temporal Emotional Modeling Approach for Speech Emotion Recognition](https://mlsanity.com/paper/2211.08233/) (2022)
- [Speaker-invariant Affective Representation Learning via Adversarial Training](https://mlsanity.com/paper/1911.01533/) (2019)
- [Speaker Normalization for Self-supervised Speech Emotion Recognition](https://mlsanity.com/paper/2202.01252/) (2022)
- [Speech Emotion Recognition via Contrastive Loss under Siamese Networks](https://mlsanity.com/paper/1910.11174/) (2019)
- [Unsupervised Speech Enhancement with speech recognition embedding and disentanglement losses](https://mlsanity.com/paper/2111.08678/) (2021)
- [Siamese Capsule Network for End-to-End Speaker Recognition In The Wild](https://mlsanity.com/paper/2009.13480/) (2020)
- [Mingling or Misalignment? Temporal Shift for Speech Emotion Recognition with Pre-trained Representations](https://mlsanity.com/paper/2302.13277/) (2023)
- [CTA-RNN: Channel and Temporal-wise Attention RNN Leveraging Pre-trained ASR Embeddings for Speech Emotion Recognition](https://mlsanity.com/paper/2203.17023/) (2022)

## Built on by (most cited first, 1 of 1)

- [CMIS-Net: A Cascaded Multi-Scale Individual Standardization Network for Backchannel Agreement Estimation](https://mlsanity.com/paper/2510.17855/) (2025)

## Abstract (arXiv)

One persistent challenge in deep learning based speech emotion recognition (SER) is the unconscious encoding of emotion-irrelevant factors (e.g., speaker or phonetic variability), which limits the generalization of SER in practical use. In this paper, we propose DSNet, a Disentangled Siamese Network with neutral calibration, to meet the demand for a more robust and explainable SER model. Specifically, we introduce an orthogonal feature disentanglement module to explicitly project the high-level representation into two distinct subspaces. Later, we propose a novel neutral calibration mechanism to encourage one subspace to capture sufficient emotion-irrelevant information. In this way, the other one can better isolate and emphasize the emotion-relevant information within speech signals. Experimental results on two popular benchmark datasets demonstrate the superiority of DSNet over various state-of-the-art methods for speaker-independent SER.