# Speaker Normalization for Self-supervised Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2202.01252 · HTML: https://mlsanity.com/paper/2202.01252/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 2 Feb 2022.

## Builds on (most cited first, 18 of 18)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders](https://mlsanity.com/paper/1910.12638/) (2019)
- [TERA: Self-Supervised Learning of Transformer Encoder Representation for Speech](https://mlsanity.com/paper/2007.06028/) (2020)
- [An Unsupervised Autoregressive Model for Speech Representation Learning](https://mlsanity.com/paper/1904.03240/) (2019)
- [Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings](https://mlsanity.com/paper/2104.03502/) (2021)
- [Unsupervised pretraining transfers well across languages](https://mlsanity.com/paper/2002.02848/) (2020)
- [Vector-Quantized Autoregressive Predictive Coding](https://mlsanity.com/paper/2005.08392/) (2020)
- [Multimodal Speech Emotion Recognition Using Audio and Text](https://mlsanity.com/paper/1810.04635/) (2018)
- [Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependencies](https://mlsanity.com/paper/2011.00406/) (2020)
- [Disentangled Speech Embeddings using Cross-modal Self-supervision](https://mlsanity.com/paper/2002.08742/) (2020)
- [Speech Emotion Recognition with Dual-Sequence LSTM Architecture](https://mlsanity.com/paper/1910.08874/) (2019)
- [Convolutional RNN: an Enhanced Model for Extracting Features from Sequential Data](https://mlsanity.com/paper/1602.05875/) (2016)
- [Speaker Attentive Speech Emotion Recognition](https://mlsanity.com/paper/2104.07288/) (2021)

## Built on by (most cited first, 14 of 14)

- [Speech Emotion Diarization: Which Emotion Appears When?](https://mlsanity.com/paper/2306.12991/) (2023)
- [Leveraging Speech PTM, Text LLM, and Emotional TTS for Speech Emotion Recognition](https://mlsanity.com/paper/2309.10294/) (2023)
- [GEmo-CLAP: Gender-Attribute-Enhanced Contrastive Language-Audio Pretraining for Accurate Speech Emotion Recognition](https://mlsanity.com/paper/2306.07848/) (2023)
- [Topological Data Analysis for Speech Processing](https://mlsanity.com/paper/2211.17223/) (2022)
- [Designing and Evaluating Speech Emotion Recognition Systems: A reality check case study with IEMOCAP](https://mlsanity.com/paper/2304.00860/) (2023)
- [MFHCA: Enhancing Speech Emotion Recognition Via Multi-Spatial Fusion and Hierarchical Cooperative Attention](https://mlsanity.com/paper/2404.13509/) (2024)
- [MSAC: Multiple Speech Attribute Control Method for Reliable Speech Emotion Recognition](https://mlsanity.com/paper/2308.04025/) (2023)
- [Speech Emotion Recognition via an Attentive Time-Frequency Neural Network](https://mlsanity.com/paper/2210.12430/) (2022)
- [Parameter Efficient Finetuning for Speech Emotion Recognition and Domain Adaptation](https://mlsanity.com/paper/2402.11747/) (2024)
- [Frame-level emotional state alignment method for speech emotion recognition](https://mlsanity.com/paper/2312.16383/) (2023)
- [GMP-TL: Gender-augmented Multi-scale Pseudo-label Enhanced Transfer Learning for Speech Emotion Recognition](https://mlsanity.com/paper/2405.02151/) (2024)
- [Adversarial Speaker Disentanglement Using Unannotated External Data for Self-supervised Representation Based Voice Conversion](https://mlsanity.com/paper/2305.09167/) (2023)
- [DSNet: Disentangled Siamese Network with Neutral Calibration for Speech Emotion Recognition](https://mlsanity.com/paper/2312.15593/) (2023)
- [Investigating Acoustic-Textual Emotional Inconsistency Information for Automatic Depression Detection](https://mlsanity.com/paper/2412.18614/) (2024)

## Abstract (arXiv)

Large speech emotion recognition datasets are hard to obtain, and small datasets may contain biases. Deep-net-based classifiers, in turn, are prone to exploit those biases and find shortcuts such as speaker characteristics. These shortcuts usually harm a model's ability to generalize. To address this challenge, we propose a gradient-based adversary learning framework that learns a speech emotion recognition task while normalizing speaker characteristics from the feature representation. We demonstrate the efficacy of our method on both speaker-independent and speaker-dependent settings and obtain new state-of-the-art results on the challenging IEMOCAP dataset.