mlsanityData to 7 Oct 2026

GTR-Voice: Articulatory Phonetics Informed Controllable Expressive Speech Synthesis

arXiv 2406.10514 ↗First posted 15 Jun 2024eess.AS cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Expressive speech synthesis aims to generate speech that captures a wide range of para-linguistic features, including emotion and articulation, though current research primarily emphasizes emotional aspects over the nuanced articulatory features mastered by professional voice actors. Inspired by this, we explore expressive speech synthesis through the lens of articulatory phonetics. Specifically, we define a framework with three dimensions: Glottalization, Tenseness, and Resonance (GTR), to guide the synthesis at the voice production level. With this framework, we record a high-quality speech dataset named GTR-Voice, featuring 20 Chinese sentences articulated by a professional voice actor across 125 distinct GTR combinations. We verify the framework and GTR annotations through automatic classification and listening tests, and demonstrate precise controllability along the GTR dimensions on two fine-tuned expressive TTS models. We open-source the dataset and TTS models.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 18 of its 44 references that are in the map. First posted on arXiv 15 Jun 2024.

Builds on

18 of 44 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  3. 2021A Survey on Neural Speech Synthesiscited by 182
  4. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  5. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
  6. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  7. 2023Audiobox: Unified Audio Generation with Natural Language Promptscited by 120
  8. 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 90
  9. 2022EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidancecited by 37
  10. 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 32
  11. 2022MsEmoTTS: Multi-scale emotion transfer, prediction, and control for emotional speech synthesiscited by 29
  12. 2022Self-supervised Context-aware Style Representation for Expressive Speech Synthesiscited by 10
6 more
  1. 2022Deep Speech Synthesis from Articulatory Representationscited by 10
  2. 2022ControlVC: Zero-Shot Voice Conversion with Time-Varying Controls on Pitch and Speedcited by 7
  3. 2023METTS: Multilingual Emotional Text-to-Speech by Cross-speaker and Cross-lingual Emotion Transfercited by 6
  4. 2023Fine-grained Emotional Control of Text-To-Speech: Learning To Rank Inter- And Intra-Class Emotion Intensitiescited by 5
  5. 2023EE-TTS: Emphatic Expressive TTS with Linguistic Informationcited by 4
  6. 2019The Theory behind Controllable Expressive Speech Synthesis: a Cross-disciplinary Approachcited by 3

Built on by

Not cited yet.