# Controllable Accented Text-to-Speech Synthesis

arXiv: https://arxiv.org/abs/2209.10804 · HTML: https://mlsanity.com/paper/2209.10804/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 70 references that are in the map.
- First posted on arXiv 22 Sep 2022.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [FastPitch: Parallel Text-to-speech with Pitch Prediction](https://mlsanity.com/paper/2006.06873/) (2020)
- [Hierarchical Generative Modeling for Controllable Speech Synthesis](https://mlsanity.com/paper/1810.07217/) (2018)
- [Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation](https://mlsanity.com/paper/2106.03153/) (2021)
- [Deep Voice: Real-time Neural Text-to-Speech](https://mlsanity.com/paper/1702.07825/) (2017)
- [Deep Voice 2: Multi-Speaker Neural Text-to-Speech](https://mlsanity.com/paper/1705.08947/) (2017)
- [The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods](https://mlsanity.com/paper/1804.04262/) (2018)
- [Learning to Speak Fluently in a Foreign Language: Multilingual Speech Synthesis and Cross-Language Voice Cloning](https://mlsanity.com/paper/1907.04448/) (2019)
- [SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model](https://mlsanity.com/paper/2104.05557/) (2021)
- [WaveFlow: A Compact Flow-based Model for Raw Audio](https://mlsanity.com/paper/1912.01219/) (2019)
- [MultiSpeech: Multi-Speaker Text to Speech with Transformer](https://mlsanity.com/paper/2006.04664/) (2020)
- [Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis](https://mlsanity.com/paper/2002.03785/) (2020)
- [Expressive TTS Training with Frame and Style Reconstruction Loss](https://mlsanity.com/paper/2008.01490/) (2020)
- [Fine-grained Emotion Strength Transfer, Control and Prediction for Emotional Speech Synthesis](https://mlsanity.com/paper/2011.08477/) (2020)
- [Multi-SpectroGAN: High-Diversity and High-Fidelity Spectrogram Generation with Adversarial Style Combination for Speech Synthesis](https://mlsanity.com/paper/2012.07267/) (2020)
- [Universal Neural Vocoding with Parallel WaveNet](https://mlsanity.com/paper/2102.01106/) (2021)
- [STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speech](https://mlsanity.com/paper/2103.09474/) (2021)
- [iEmoTTS: Toward Robust Cross-Speaker Emotion Transfer and Control for Speech Synthesis based on Disentanglement between Prosody and Timbre](https://mlsanity.com/paper/2206.14866/) (2022)
- [FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesis](https://mlsanity.com/paper/2106.15123/) (2021)
- [Emotional Prosody Control for Speech Generation](https://mlsanity.com/paper/2111.04730/) (2021)
- [Hierarchical prosody modeling and control in non-autoregressive parallel neural TTS](https://mlsanity.com/paper/2110.02952/) (2021)

## Built on by (most cited first, 2 of 2)

- [Accented Text-to-Speech Synthesis with a Conditional Variational Autoencoder](https://mlsanity.com/paper/2211.03316/) (2022)
- [A Pilot Study of GSLM-based Simulation of Foreign Accentuation Only Using Native Speech Corpora](https://mlsanity.com/paper/2407.11370/) (2024)

## Abstract (arXiv)

Accented text-to-speech (TTS) synthesis seeks to generate speech with an accent (L2) as a variant of the standard version (L1). Accented TTS synthesis is challenging as L2 is different from L1 in both in terms of phonetic rendering and prosody pattern. Furthermore, there is no easy solution to the control of the accent intensity in an utterance. In this work, we propose a neural TTS architecture, that allows us to control the accent and its intensity during inference. This is achieved through three novel mechanisms, 1) an accent variance adaptor to model the complex accent variance with three prosody controlling factors, namely pitch, energy and duration; 2) an accent intensity modeling strategy to quantify the accent intensity; 3) a consistency constraint module to encourage the TTS system to render the expected accent intensity at a fine level. Experiments show that the proposed system attains superior performance to the baseline models in terms of accent rendering and intensity control. To our best knowledge, this is the first study of accented TTS synthesis with explicit intensity control.