# Semi-supervised cross-lingual speech emotion recognition

arXiv: https://arxiv.org/abs/2207.06767 · HTML: https://mlsanity.com/paper/2207.06767/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 49 references that are in the map.
- First posted on arXiv 14 Jul 2022.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Joint Optimization Framework for Learning with Noisy Labels](https://mlsanity.com/paper/1803.11364/) (2018)
- [BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation](https://mlsanity.com/paper/2103.06695/) (2021)
- [Domain Adversarial for Acoustic Emotion Recognition](https://mlsanity.com/paper/1804.07690/) (2018)
- [SERAB: A multi-lingual benchmark for speech emotion recognition](https://mlsanity.com/paper/2110.03414/) (2021)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Cross-lingual and Multilingual Speech Emotion Recognition on English and French](https://mlsanity.com/paper/1803.00357/) (2018)
- [Unsupervised Adversarial Domain Adaptation for Cross-Lingual Speech Emotion Recognition](https://mlsanity.com/paper/1907.06083/) (2019)
- [BYOL-S: Learning Self-supervised Speech Representations by Bootstrapping](https://mlsanity.com/paper/2206.12038/) (2022)
- [Towards Speech Emotion Recognition "in the wild" using Aggregated Corpora and Deep Multi-Task Learning](https://mlsanity.com/paper/1708.03920/) (2017)
- [Transferable Positive/Negative Speech Emotion Recognition via Class-wise Adversarial Domain Adaptation](https://mlsanity.com/paper/1810.12782/) (2018)
- [Unsupervised Cross-Lingual Speech Emotion Recognition Using DomainAdversarial Neural Network](https://mlsanity.com/paper/2012.11174/) (2020)
- [Unsupervised Cross-Lingual Speech Emotion Recognition Using Pseudo Multilabel](https://mlsanity.com/paper/2108.08663/) (2021)
- [Towards Transferable Speech Emotion Representation: On loss functions for cross-lingual latent representations](https://mlsanity.com/paper/2203.14865/) (2022)

## Built on by (most cited first, 3 of 3)

- [Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection](https://mlsanity.com/paper/2409.10985/) (2024)
- [SER Evals: In-domain and Out-of-domain Benchmarking for Speech Emotion Recognition](https://mlsanity.com/paper/2408.07851/) (2024)
- [A Persian ASR-based SER: Modification of Sharif Emotional Speech Database and Investigation of Persian Text Corpora](https://mlsanity.com/paper/2211.09956/) (2022)

## Abstract (arXiv)

Performance in Speech Emotion Recognition (SER) on a single language has increased greatly in the last few years thanks to the use of deep learning techniques. However, cross-lingual SER remains a challenge in real-world applications due to two main factors: the first is the big gap among the source and the target domain distributions; the second factor is the major availability of unlabeled utterances in contrast to the labeled ones for the new language. Taking into account previous aspects, we propose a Semi-Supervised Learning (SSL) method for cross-lingual emotion recognition when only few labeled examples in the target domain (i.e. the new language) are available. Our method is based on a Transformer and it adapts to the new domain by exploiting a pseudo-labeling strategy on the unlabeled utterances. In particular, the use of a hard and soft pseudo-labels approach is investigated. We thoroughly evaluate the performance of the proposed method in a speaker-independent setup on both the source and the new language and show its robustness across five languages belonging to different linguistic strains. The experimental findings indicate that the unweighted accuracy is increased by an average of 40% compared to state-of-the-art methods.