# Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection

arXiv: https://arxiv.org/abs/2409.10985 · HTML: https://mlsanity.com/paper/2409.10985/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 45 references that are in the map.
- First posted on arXiv 17 Sep 2024.

## Builds on (most cited first, 18 of 18)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation](https://mlsanity.com/paper/2312.15185/) (2023)
- [Seamless: Multilingual Expressive and Streaming Speech Translation](https://mlsanity.com/paper/2312.05187/) (2023)
- [Out-of-domain Detection for Natural Language Understanding in Dialog Systems](https://mlsanity.com/paper/1909.03862/) (2019)
- [EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmark](https://mlsanity.com/paper/2406.07162/) (2024)
- [Leveraging Speech PTM, Text LLM, and Emotional TTS for Speech Emotion Recognition](https://mlsanity.com/paper/2309.10294/) (2023)
- [Augmenting Generative Adversarial Networks for Speech Emotion Recognition](https://mlsanity.com/paper/2005.08447/) (2020)
- [Cross-lingual and Multilingual Speech Emotion Recognition on English and French](https://mlsanity.com/paper/1803.00357/) (2018)
- [Semi-Supervised Speech Emotion Recognition with Ladder Networks](https://mlsanity.com/paper/1905.02921/) (2019)
- [Generative Emotional AI for Speech Emotion Recognition: The Case for Synthetic Emotional Speech Augmentation](https://mlsanity.com/paper/2301.03751/) (2023)
- [SponTTS: modeling and transferring spontaneous style for TTS](https://mlsanity.com/paper/2311.07179/) (2023)
- [Large Raw Emotional Dataset with Aggregation Mechanism](https://mlsanity.com/paper/2212.12266/) (2022)
- [Semi-supervised cross-lingual speech emotion recognition](https://mlsanity.com/paper/2207.06767/) (2022)
- [Deep Implicit Distribution Alignment Networks for Cross-Corpus Speech Emotion Recognition](https://mlsanity.com/paper/2302.08921/) (2023)

## Built on by (most cited first, 10 of 10)

- [Fake-Mamba: Real-Time Speech Deepfake Detection Using Bidirectional Mamba as Self-Attention's Alternative](https://mlsanity.com/paper/2508.09294/) (2025)
- [Mitigating Subgroup Disparities in Multi-Label Speech Emotion Recognition: A Pseudo-Labeling and Unsupervised Learning Approach](https://mlsanity.com/paper/2505.14449/) (2025)
- [EMO-Debias: Benchmarking Gender Debiasing Techniques in Multi-Label Speech Emotion Recognition](https://mlsanity.com/paper/2506.04652/) (2025)
- [CantoASR: Prosody-Aware ASR-LALM Collaboration for Low-Resource Cantonese](https://mlsanity.com/paper/2511.04139/) (2025)
- [Meta-PerSER: Few-Shot Listener Personalized Speech Emotion Recognition via Meta-learning](https://mlsanity.com/paper/2505.16220/) (2025)
- [Learning More with Less: Self-Supervised Approaches for Low-Resource Speech Emotion Recognition](https://mlsanity.com/paper/2506.02059/) (2025)
- [AMRD: Adaptive Multi-Teacher Relational Distillation for Lightweight Speech Emotion Recognition](https://mlsanity.com/paper/2607.25289/) (2026)
- [CO-VADA: A Confidence-Oriented Voice Augmentation Debiasing Approach for Fair Speech Emotion Recognition](https://mlsanity.com/paper/2506.06071/) (2025)
- [EMO-TTA: Improving Test-Time Adaptation of Audio-Language Models for Speech Emotion Recognition](https://mlsanity.com/paper/2509.25495/) (2025)
- [Plug-and-Play Emotion Graphs for Compositional Prompting in Zero-Shot Speech Emotion Recognition](https://mlsanity.com/paper/2509.25458/) (2025)

## Abstract (arXiv)

Speech Emotion Recognition (SER) is a crucial component in developing general-purpose AI agents capable of natural human-computer interaction. However, building robust multilingual SER systems remains challenging due to the scarcity of labeled data in languages other than English and Chinese. In this paper, we propose an approach to enhance SER performance in low SER resource languages by leveraging data from high-resource languages. Specifically, we employ expressive Speech-to-Speech translation (S2ST) combined with a novel bootstrapping data selection pipeline to generate labeled data in the target language. Extensive experiments demonstrate that our method is both effective and generalizable across different upstream models and languages. Our results suggest that this approach can facilitate the development of more scalable and robust multilingual SER systems.