# CASEIN: Cascading Explicit and Implicit Control for Fine-grained Emotion Intensity Regulation

arXiv: https://arxiv.org/abs/2307.00020 · HTML: https://mlsanity.com/paper/2307.00020/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 23 references that are in the map.
- First posted on arXiv 27 Jun 2023.

## Builds on (most cited first, 18 of 18)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [Emotional Voice Conversion: Theory, Databases and ESD](https://mlsanity.com/paper/2105.14762/) (2021)
- [Robust and fine-grained prosody control of end-to-end speech synthesis](https://mlsanity.com/paper/1811.02122/) (2018)
- [EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidance](https://mlsanity.com/paper/2211.09496/) (2022)
- [Emotional End-to-End Neural Speech Synthesizer](https://mlsanity.com/paper/1711.05447/) (2017)
- [Fine-grained Emotion Strength Transfer, Control and Prediction for Emotional Speech Synthesis](https://mlsanity.com/paper/2011.08477/) (2020)
- [MsEmoTTS: Multi-scale emotion transfer, prediction, and control for emotional speech synthesis](https://mlsanity.com/paper/2201.06460/) (2022)
- [Emotional speech synthesis with rich and granularized control](https://mlsanity.com/paper/1911.01635/) (2019)
- [Speech Synthesis with Mixed Emotions](https://mlsanity.com/paper/2208.05890/) (2022)
- [Fine-grained robust prosody transfer for single-speaker neural text-to-speech](https://mlsanity.com/paper/1907.02479/) (2019)
- [Towards Multi-Scale Style Control for Expressive Speech Synthesis](https://mlsanity.com/paper/2104.03521/) (2021)
- [Uncovering Latent Style Factors for Expressive Speech Synthesis](https://mlsanity.com/paper/1711.00520/) (2017)
- [Deep Factorization for Speech Signal](https://mlsanity.com/paper/1706.01777/) (2017)
- [StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis](https://mlsanity.com/paper/2110.03156/) (2021)

## Abstract (arXiv)

Existing fine-grained intensity regulation methods rely on explicit control through predicted emotion probabilities. However, these high-level semantic probabilities are often inaccurate and unsmooth at the phoneme level, leading to bias in learning. Especially when we attempt to mix multiple emotion intensities for specific phonemes, resulting in markedly reduced controllability and naturalness of the synthesis. To address this issue, we propose the CAScaded Explicit and Implicit coNtrol framework (CASEIN), which leverages accurate disentanglement of emotion manifolds from the reference speech to learn the implicit representation at a lower semantic level. This representation bridges the semantical gap between explicit probabilities and the synthesis model, reducing bias in learning. In experiments, our CASEIN surpasses existing methods in both controllability and naturalness. Notably, we are the first to achieve fine-grained control over the mixed intensity of multiple emotions.