mlsanityData to 7 Oct 2026

A Layer-Anchoring Strategy for Enhancing Cross-Lingual Speech Emotion Recognition

arXiv 2407.04966 ↗First posted 6 Jul 2024cs.SD cs.LG eess.AScited by 6

Not on the map. Why?

Abstract (arXiv)

Cross-lingual speech emotion recognition (SER) is important for a wide range of everyday applications. While recent SER research relies heavily on large pretrained models for emotion training, existing studies often concentrate solely on the final transformer layer of these models. However, given the task-specific nature and hierarchical architecture of these models, each transformer layer encapsulates different levels of information. Leveraging this hierarchical structure, our study focuses on the information embedded across different layers. Through an examination of layer feature similarity across different languages, we propose a novel strategy called a layer-anchoring mechanism to facilitate emotion transfer in cross-lingual SER tasks. Our approach is evaluated using two distinct language affective corpora (MSP-Podcast and BIIC-Podcast), achieving a best UAR performance of 60.21% on the BIIC-podcast corpus. The analysis uncovers interesting insights into the behavior of popular pretrained models.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 15 of its 25 references that are in the map. First posted on arXiv 6 Jul 2024.

Builds on

15 of 25 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  5. 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 158
  6. 2020Exploring wav2vec 2.0 on speaker verification and language identificationcited by 93
  7. 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
  8. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 70
  9. 2021Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognitioncited by 47
  10. 2022Exploration of A Self-Supervised Speech Model: A Study on Emotional Corporacited by 31
  11. 2023PEFT-SER: On the Use of Parameter Efficient Transfer Learning Approaches For Speech Emotion Recognition Using Pre-trained Speech Modelscited by 17
  12. 2020Neural Representations for Modeling Variation in Speechcited by 14
3 more
  1. 2022An attention-based backend allowing efficient fine-tuning of transformer models for speaker verificationcited by 14
  2. 2023A Study on Incorporating Whisper for Robust Speech Assessmentcited by 11
  3. 2023Speaker and Language Change Detection using Wav2vec2 and Whispercited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 1.

6 papers.

  1. 2024Cross-Lingual Speech Emotion Recognition: Humans vs. Self-Supervised Modelscited by 3
  2. 2025Is It Still Fair? Investigating Gender Fairness in Cross-Corpus Speech Emotion Recognitioncited by 3
  3. 2024Describe Where You Are: Improving Noise-Robustness for Speech Emotion Recognition with Text Description of the Environmentcited by 1
  4. 2024Mouth Articulation-Based Anchoring for Improved Cross-Corpus Speech Emotion Recognitioncited by 1
  5. 2025Speaker Style-Aware Phoneme Anchoring for Improved Cross-Lingual Speech Emotion Recognitioncited by 0
  6. 2026Emotion as a Distribution: Joint Valence-Arousal Probability Learning for Speaker-Independent Multimodal Emotion Recognitioncited by 0