Emotional Text-To-Speech Based on Mutual-Information-Guided Emotion-Timbre Disentanglement
Not on the map. Why?
Abstract (arXiv)
Current emotional Text-To-Speech (TTS) and style transfer methods rely on reference encoders to control global style or emotion vectors, but do not capture nuanced acoustic details of the reference speech. To this end, we propose a novel emotional TTS method that enables fine-grained phoneme-level emotion embedding prediction while disentangling intrinsic attributes of the reference speech. The proposed method employs a style disentanglement method to guide two feature extractors, reducing mutual information between timbre and emotion features, and effectively separating distinct style components from the reference speech. Experimental results demonstrate that our method outperforms baseline TTS systems in generating natural and emotionally rich speech. This work highlights the potential of disentangled and fine-grained representations in advancing the quality and flexibility of emotional TTS systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 23 references that are in the map. First posted on arXiv 2 Oct 2025.
Builds on
18 of 23 references.
References · 18
- 2017Attention Is All You Needcited by 59,932
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
- 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
- 2018MINE: Mutual Information Neural Estimationcited by 680
- 2022High Fidelity Neural Audio Compressioncited by 673
- 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
- 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
- 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 113
6 more
- 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
- 2018Neural Speech Synthesis with Transformer Networkcited by 14
- 2020Unsupervised Style and Content Separation by Minimizing Mutual Information for Speech Synthesiscited by 11
- 2022Self-Attentive Pooling for Efficient Deep Learningcited by 8
- 2018Scalable Factorized Hierarchical Variational Autoencoder Trainingcited by 5
- 2023DC CoMix TTS: An End-to-End Expressive TTS with Discrete Code Collaborated with Mixercited by 1
Built on by
Not cited yet.