# Speaker Style-Aware Phoneme Anchoring for Improved Cross-Lingual Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2509.20373 · HTML: https://mlsanity.com/paper/2509.20373/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 35 references that are in the map.
- First posted on arXiv 19 Sep 2025.

## Builds on (most cited first, 12 of 12)

- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation](https://mlsanity.com/paper/2303.11089/) (2023)
- [FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion](https://mlsanity.com/paper/2210.15418/) (2022)
- [VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversion](https://mlsanity.com/paper/2106.10132/) (2021)
- [Domain Adversarial for Acoustic Emotion Recognition](https://mlsanity.com/paper/1804.07690/) (2018)
- [Improving Cross-Corpus Speech Emotion Recognition with Adversarial Discriminative Domain Generalization (ADDoG)](https://mlsanity.com/paper/1903.12094/) (2019)
- [Semi-Supervised Speech Emotion Recognition with Ladder Networks](https://mlsanity.com/paper/1905.02921/) (2019)
- [Compact Graph Architecture for Speech Emotion Recognition](https://mlsanity.com/paper/2008.02063/) (2020)
- [Revealing Emotional Clusters in Speaker Embeddings: A Contrastive Learning Strategy for Speech Emotion Recognition](https://mlsanity.com/paper/2401.11017/) (2024)
- [A Layer-Anchoring Strategy for Enhancing Cross-Lingual Speech Emotion Recognition](https://mlsanity.com/paper/2407.04966/) (2024)
- [Mouth Articulation-Based Anchoring for Improved Cross-Corpus Speech Emotion Recognition](https://mlsanity.com/paper/2412.19909/) (2024)

## Abstract (arXiv)

Cross-lingual speech emotion recognition (SER) remains a challenging task due to differences in phonetic variability and speaker-specific expressive styles across languages. Effectively capturing emotion under such diverse conditions requires a framework that can align the externalization of emotions across different speakers and languages. To address this problem, we propose a speaker-style aware phoneme anchoring framework that aligns emotional expression at the phonetic and speaker levels. Our method builds emotion-specific speaker communities via graph-based clustering to capture shared speaker traits. Using these groups, we apply dual-space anchoring in speaker and phonetic spaces to enable better emotion transfer across languages. Evaluations on the MSP-Podcast (English) and BIIC-Podcast (Taiwanese Mandarin) corpora demonstrate improved generalization over competitive baselines and provide valuable insights into the commonalities in cross-lingual emotion representation.