mlsanityData to 14 Oct 2026

Expressivity and Speech Synthesis

arXiv 2404.19363 ↗First posted 30 Apr 2024cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

Imbuing machines with the ability to talk has been a longtime pursuit of artificial intelligence (AI) research. From the very beginning, the community has not only aimed to synthesise high-fidelity speech that accurately conveys the semantic meaning of an utterance, but also to colour it with inflections that cover the same range of affective expressions that humans are capable of. After many years of research, it appears that we are on the cusp of achieving this when it comes to single, isolated utterances. This unveils an abundance of potential avenues to explore when it comes to combining these single utterances with the aim of synthesising more complex, longer-term behaviours. In the present chapter, we outline the methodological advances that brought us so far and sketch out the ongoing efforts to reach that coveted next level of artificial expressivity. We also discuss the societal implications coupled with rapidly advancing expressive speech synthesis (ESS) technology and highlight ways to mitigate those risks and ensure the alignment of ESS capabilities with ethical norms.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 29 of its 103 references that are in the map. First posted on arXiv 30 Apr 2024.

Builds on

29 of 103 references.

References · 29

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  4. 2023Consistency Modelscited by 1,822
  5. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  6. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  7. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 490
  8. 2020Artificial Intelligence, Values and Alignmentcited by 448
  9. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
  10. 2016Tutorial on Variational Autoencoderscited by 409
  11. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 259
  12. 2022A Survey on Generative Diffusion Modelcited by 240
17 more
  1. 2017Neural Style Transfer: A Reviewcited by 199
  2. 2021A Survey on Neural Speech Synthesiscited by 184
  3. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 173
  4. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 149
  5. 2023UniAudio: An Audio Foundation Model Toward Universal Audio Generationcited by 127
  6. 2022ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wildcited by 121
  7. 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 110
  8. 2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 48
  9. 2019What Makes a Good Conversation? Challenges in Designing Truly Conversational Agentscited by 42
  10. 2022Emotion Intensity and its Control for Emotional Voice Conversioncited by 35
  11. 2022An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Eracited by 33
  12. 2023PromptTTS++: Controlling Speaker Identity in Prompt-Based Text-to-Speech Using Natural Language Descriptionscited by 25
  13. 2017Affective Neural Response Generationcited by 17
  14. 2022The ICML 2022 Expressive Vocalizations Workshop and Competition: Recognizing, Generating, and Personalizing Vocal Burstscited by 16
  15. 2023EMOCONV-DIFF: Diffusion-based Speech Emotion Conversion for Non-parallel and In-the-wild Datacited by 7
  16. 2022Alexa as an Active Listener: How Backchanneling Can Elicit Self-Disclosure and Promote User Experiencecited by 6
  17. 2019Does the Lombard Effect Improve Emotional Communication in Noise? - Analysis of Emotional Speech Acted in Noise -cited by 1

Built on by

Cited by year

Citing papers per year: 2024: 3.

3 papers.

  1. 2024SLIM: Style-Linguistics Mismatch Model for Generalized Audio Deepfake Detectioncited by 8
  2. 2024Emotional Dimension Control in Language Model-Based Text-to-Speech: Spanning a Broad Spectrum of Human Emotionscited by 8
  3. 2024Enhancing Emotional Text-to-Speech Controllability with Natural Language Guidance through Contrastive Learning and Diffusion Modelscited by 3