mlsanityData to 7 Oct 2026

Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection

arXiv 2409.10985 ↗First posted 17 Sep 2024eess.AS cs.CL cs.SDcited by 10

Not on the map. Why?

Abstract (arXiv)

Speech Emotion Recognition (SER) is a crucial component in developing general-purpose AI agents capable of natural human-computer interaction. However, building robust multilingual SER systems remains challenging due to the scarcity of labeled data in languages other than English and Chinese. In this paper, we propose an approach to enhance SER performance in low SER resource languages by leveraging data from high-resource languages. Specifically, we employ expressive Speech-to-Speech translation (S2ST) combined with a novel bootstrapping data selection pipeline to generate labeled data in the target language. Extensive experiments demonstrate that our method is both effective and generalizable across different upstream models and languages. Our results suggest that this approach can facilitate the development of more scalable and robust multilingual SER systems.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 18 of its 45 references that are in the map. First posted on arXiv 17 Sep 2024.

Builds on

18 of 45 references.

References · 18

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  5. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  6. 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 196
  7. 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 190
  8. 2019Out-of-domain Detection for Natural Language Understanding in Dialog Systemscited by 62
  9. 2024EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmarkcited by 39
  10. 2023Leveraging Speech PTM, Text LLM, and Emotional TTS for Speech Emotion Recognitioncited by 12
  11. 2020Augmenting Generative Adversarial Networks for Speech Emotion Recognitioncited by 11
  12. 2018Cross-lingual and Multilingual Speech Emotion Recognition on English and Frenchcited by 10
6 more
  1. 2019Semi-Supervised Speech Emotion Recognition with Ladder Networkscited by 9
  2. 2023Generative Emotional AI for Speech Emotion Recognition: The Case for Synthetic Emotional Speech Augmentationcited by 8
  3. 2023SponTTS: modeling and transferring spontaneous style for TTScited by 7
  4. 2022Large Raw Emotional Dataset with Aggregation Mechanismcited by 5
  5. 2022Semi-supervised cross-lingual speech emotion recognitioncited by 3
  6. 2023Deep Implicit Distribution Alignment Networks for Cross-Corpus Speech Emotion Recognitioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 1.

10 papers.

  1. 2025Fake-Mamba: Real-Time Speech Deepfake Detection Using Bidirectional Mamba as Self-Attention's Alternativecited by 6
  2. 2025Mitigating Subgroup Disparities in Multi-Label Speech Emotion Recognition: A Pseudo-Labeling and Unsupervised Learning Approachcited by 5
  3. 2025EMO-Debias: Benchmarking Gender Debiasing Techniques in Multi-Label Speech Emotion Recognitioncited by 3
  4. 2025CantoASR: Prosody-Aware ASR-LALM Collaboration for Low-Resource Cantonesecited by 3
  5. 2025Meta-PerSER: Few-Shot Listener Personalized Speech Emotion Recognition via Meta-learningcited by 2
  6. 2025Learning More with Less: Self-Supervised Approaches for Low-Resource Speech Emotion Recognitioncited by 2
  7. 2026AMRD: Adaptive Multi-Teacher Relational Distillation for Lightweight Speech Emotion Recognitioncited by 2
  8. 2025CO-VADA: A Confidence-Oriented Voice Augmentation Debiasing Approach for Fair Speech Emotion Recognitioncited by 1
  9. 2025EMO-TTA: Improving Test-Time Adaptation of Audio-Language Models for Speech Emotion Recognitioncited by 1
  10. 2025Plug-and-Play Emotion Graphs for Compositional Prompting in Zero-Shot Speech Emotion Recognitioncited by 0