Towards Zero-Shot Text-To-Speech for Arabic Dialects
Not on the map. Why?
Abstract (arXiv)
Zero-shot multi-speaker text-to-speech (ZS-TTS) systems have advanced for English, however, it still lags behind due to insufficient resources. We address this gap for Arabic, a language of more than 450 million native speakers, by first adapting a sizeable existing dataset to suit the needs of speech synthesis. Additionally, we employ a set of Arabic dialect identification models to explore the impact of pre-defined dialect labels on improving the ZS-TTS model in a multi-dialect setting. Subsequently, we fine-tune the XTTSfootnotehttps://docs.coqui.ai/en/latest/models/xtts.htmlfootnotehttps://medium.com/machine-learns/xtts-v2-new-version-of-the-open-source-text-to-speech-model-af73914db81ffootnotehttps://medium.com/@erogol/xtts-v1-techincal-notes-eb83ff05bdc model, an open-source architecture. We then evaluate our models on a dataset comprising 31 unseen speakers and an in-house dialectal dataset. Our automated and human evaluation results show convincing performance while capable of generating dialectal speech. Our study highlights significant potential for improvements in this emerging area of research in Arabic.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 21 of its 31 references that are in the map. First posted on arXiv 24 Jun 2024.
Builds on
21 of 31 references.
References · 21
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
- 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
- 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
- 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
- 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 166
- 2018Neural Voice Cloning with a Few Samplescited by 100
- 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
9 more
- 2019Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddingscited by 73
- 2021SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Modelcited by 56
- 2021QASR: QCRI Aljazeera Speech Resource -- A Large Scale Annotated Arabic Speech Corpuscited by 38
- 2022Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Datacited by 35
- 2022Grad-StyleSpeech: Any-speaker Adaptive Text-to-Speech Synthesis with Diffusion Modelscited by 34
- 2020Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embeddingcited by 27
- 2022FRCRN: Boosting Feature Representation using Frequency Recurrence for Monaural Speech Enhancementcited by 24
- 2023ClArTTS: An Open-Source Classical Arabic Text-to-Speech Corpuscited by 14
- 2023ArTST: Arabic Text and Speech Transformercited by 14
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.