# FMSD-TTS: Few-shot Multi-Speaker Multi-Dialect Text-to-Speech Synthesis for "U-Tsang, Amdo and Kham Speech Dataset Generation

arXiv: https://arxiv.org/abs/2505.14351 · HTML: https://mlsanity.com/paper/2505.14351/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 43 references that are in the map.
- First posted on arXiv 20 May 2025.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching](https://mlsanity.com/paper/2410.06885/) (2024)
- [BigVGAN: A Universal Neural Vocoder with Large-Scale Training](https://mlsanity.com/paper/2206.04658/) (2022)
- [NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers](https://mlsanity.com/paper/2304.09116/) (2023)
- [XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model](https://mlsanity.com/paper/2406.04904/) (2024)
- [Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis](https://mlsanity.com/paper/2306.00814/) (2023)
- [NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality](https://mlsanity.com/paper/2205.04421/) (2022)
- [Matcha-TTS: A fast TTS architecture with conditional flow matching](https://mlsanity.com/paper/2309.03199/) (2023)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Masking](https://mlsanity.com/paper/2303.00332/) (2023)
- [AdaSpeech: Adaptive Text to Speech for Custom Voice](https://mlsanity.com/paper/2103.00993/) (2021)
- [AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarios](https://mlsanity.com/paper/2204.00436/) (2022)
- [VoxBlink2: A 100K+ Speaker Recognition Corpus and the Open-Set Speaker-Identification Benchmark](https://mlsanity.com/paper/2407.11510/) (2024)
- [VITS2: Improving Quality and Efficiency of Single-Stage Text-to-Speech with Adversarial Learning and Architecture Design](https://mlsanity.com/paper/2307.16430/) (2023)
- [AdaSpeech 3: Adaptive Text to Speech for Spontaneous Style](https://mlsanity.com/paper/2107.02530/) (2021)
- [CLAP: Isolating Content from Style through Contrastive Learning with Augmented Prompts](https://mlsanity.com/paper/2311.16445/) (2023)
- [DurFlex-EVC: Duration-Flexible Emotional Voice Conversion Leveraging Discrete Representations without Text Alignment](https://mlsanity.com/paper/2401.08095/) (2024)
- [Generalizable Zero-Shot Speaker Adaptive Speech Synthesis with Disentangled Representations](https://mlsanity.com/paper/2308.13007/) (2023)
- [Personalized Lightweight Text-to-Speech: Voice Cloning with Adaptive Structured Pruning](https://mlsanity.com/paper/2303.11816/) (2023)
- [Pruning Self-Attention for Zero-Shot Multi-Speaker Text-to-Speech](https://mlsanity.com/paper/2308.14909/) (2023)
- [ELF: Encoding Speaker-Specific Latent Speech Feature for Speech Synthesis](https://mlsanity.com/paper/2311.11745/) (2023)

## Built on by (most cited first, 3 of 3)

- [TIBSTC-CoT: A Multi-Domain Instruction Dataset for Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2508.01977/) (2025)
- [Tibetan Language and AI: A Comprehensive Survey of Resources, Methods and Challenges](https://mlsanity.com/paper/2510.19144/) (2025)
- [Tibetan-TTS:Low-Resource Tibetan Speech Synthesis with Large Model Adaptation](https://mlsanity.com/paper/2605.02496/) (2026)

## Abstract (arXiv)

Tibetan is a low-resource language with minimal parallel speech corpora spanning its three major dialects-"U-Tsang, Amdo, and Kham-limiting progress in speech modeling. To address this issue, we propose FMSD-TTS, a few-shot, multi-speaker, multi-dialect text-to-speech framework that synthesizes parallel dialectal speech from limited reference audio and explicit dialect labels. Our method features a novel speaker-dialect fusion module and a Dialect-Specialized Dynamic Routing Network (DSDR-Net) to capture fine-grained acoustic and linguistic variations across dialects while preserving speaker identity. Extensive objective and subjective evaluations demonstrate that FMSD-TTS significantly outperforms baselines in both dialectal expressiveness and speaker similarity. We further validate the quality and utility of the synthesized speech through a challenging speech-to-speech dialect conversion task. Our contributions include: (1) a novel few-shot TTS system tailored for Tibetan multi-dialect speech synthesis, (2) the public release of a large-scale synthetic Tibetan speech corpus generated by FMSD-TTS, and (3) an open-source evaluation toolkit for standardized assessment of speaker similarity, dialect consistency, and audio quality.