mlsanityData to 7 Oct 2026

Bailing-TTS: Chinese Dialectal Speech Synthesis Towards Human-like Spontaneous Representation

arXiv 2408.00284 ↗First posted 1 Aug 2024cs.CL cs.SD eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

Large-scale text-to-speech (TTS) models have made significant progress recently.However, they still fall short in the generation of Chinese dialectal speech. Toaddress this, we propose Bailing-TTS, a family of large-scale TTS models capable of generating high-quality Chinese dialectal speech. Bailing-TTS serves as a foundation model for Chinese dialectal speech generation. First, continual semi-supervised learning is proposed to facilitate the alignment of text tokens and speech tokens. Second, the Chinese dialectal representation learning is developed using a specific transformer architecture and multi-stage training processes. With the proposed design of novel network architecture and corresponding strategy, Bailing-TTS is able to generate Chinese dialectal speech from text effectively and efficiently. Experiments demonstrate that Bailing-TTS generates Chinese dialectal speech towards human-like spontaneous representation. Readers are encouraged to listen to demos at urlhttps://c9412600.github.io/blttstechreport/index.html.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 34 of its 38 references that are in the map. First posted on arXiv 1 Aug 2024.

Builds on

34 of 38 references.

References · 34

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  4. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  6. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  7. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  8. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  9. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  10. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  11. 2023Language Model Beats Diffusion -- Tokenizer is Key to Visual Generationcited by 633
  12. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
22 more
  1. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  2. 2021A White Paper on Neural Network Quantizationcited by 453
  3. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  4. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  5. 2017Deep Reinforcement Learning: An Overviewcited by 327
  6. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 233
  7. 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 166
  8. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  9. 2023SoundStorm: Efficient Parallel Audio Generationcited by 111
  10. 2023FunASR: A Fundamental End-to-End Speech Recognition Toolkitcited by 92
  11. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 74
  12. 2023SpeechX: Neural Codec Language Model as a Versatile Speech Transformercited by 64
  13. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Biascited by 51
  14. 2024ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reorderingcited by 50
  15. 2024SpeechAlign: Aligning Speech Generation to Human Preferencescited by 45
  16. 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversioncited by 38
  17. 2020DiDiSpeech: A Large Scale Mandarin Speech Corpuscited by 33
  18. 2024MusicRL: Aligning Music Generation to Human Preferencescited by 32
  19. 2023E3 TTS: Easy End-to-End Diffusion-based Text to Speechcited by 30
  20. 2022ResGrad: Residual Denoising Diffusion Probabilistic Models for Text to Speechcited by 21
  21. 2023LM-VC: Zero-shot Voice Conversion via Speech Generation based on Language Modelscited by 18
  22. 2024decoupleQ: Towards 2-bit Post-Training Uniform Quantization via decoupling Parameters into Integer and Floating Pointscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Surveycited by 22
  2. 2025Task Vector in TTS: Toward Emotionally Expressive Dialectal Speech Synthesiscited by 2
  3. 2026PolySpeech-100: A Large-Scale Benchmark for Speech Understanding Across 100+ Languages and Dialectscited by 0