mlsanityData to 7 Oct 2026

Speak, Read and Prompt: High-Fidelity Text-to-Speech with Minimal Supervision

arXiv 2302.03540 ↗First posted 7 Feb 2023cs.SD eess.AScited by 167

Not on the map. Why?

Cited by 23 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce SPEAR-TTS, a multi-speaker text-to-speech (TTS) system that can be trained with minimal supervision. By combining two types of discrete speech representations, we cast TTS as a composition of two sequence-to-sequence tasks: from text to high-level semantic tokens (akin to "reading") and from semantic tokens to low-level acoustic tokens ("speaking"). Decoupling these two tasks enables training of the "speaking" module using abundant audio-only data, and unlocks the highly efficient combination of pretraining and backtranslation to reduce the need for parallel data when training the "reading" component. To control the speaker identity, we adopt example prompting, which allows SPEAR-TTS to generalize to unseen speakers using only a short sample of 3 seconds, without any explicit speaker representation or speaker-id labels. Our experiments demonstrate that SPEAR-TTS achieves a character error rate that is competitive with state-of-the-art methods using only 15 minutes of parallel data, while matching ground-truth speech in terms of naturalness and acoustic quality, as measured in subjective tests.

Key facts. Cited by 167 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Feb 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 28; 2024: 73; 2025: 46; 2026: 20.

167 papers.

Sort:

By area:

  1. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  2. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 333
  3. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
  4. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 233
  5. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
  6. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 204
  7. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
  8. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 142
  9. 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 124
  10. 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 119
  11. 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 118
  12. 2025On The Landscape of Spoken Language Models: A Comprehensive Surveycited by 116
38 more
  1. 2023SoundStorm: Efficient Parallel Audio Generationcited by 111
  2. 2025Step-Audio 2 Technical Reportcited by 101
  3. 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 93
  4. 2023SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Modelscited by 85
  5. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 74
  6. 2024Proactive Detection of Voice Cloning with Localized Watermarkingcited by 67
  7. 2023SpeechX: Neural Codec Language Model as a Versatile Speech Transformercited by 64
  8. 2025DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generationcited by 56
  9. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Biascited by 51
  10. 2023Efficient Neural Music Generationcited by 50
  11. 2024ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reorderingcited by 50
  12. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
  13. 2023WavMark: Watermarking for Audio Generationcited by 41
  14. 2024VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignmentcited by 37
  15. 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
  16. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesiscited by 34
  17. 2024Foundation Models for Music: A Surveycited by 34
  18. 2024MusicRL: Aligning Music Generation to Human Preferencescited by 32
  19. 2024DASB - Discrete Audio and Speech Benchmarkcited by 32
  20. 2024DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factorscited by 31
  21. 2023Make-A-Voice: Unified Voice Synthesis With Discrete Representationcited by 27
  22. 2024RALL-E: Robust Codec Language Modeling with Chain-of-Thought Prompting for Text-to-Speech Synthesiscited by 26
  23. 2025Streaming Sequence-to-Sequence Learning with Delayed Streams Modelingcited by 24
  24. 2024SimpleSpeech: Towards Simple and Efficient Text-to-Speech with Scalar Latent Transformer Diffusion Modelscited by 23
  25. 2023From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusioncited by 22
  26. 2024FlashSpeech: Efficient Zero-Shot Speech Synthesiscited by 22
  27. 2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Surveycited by 22
  28. 2023SpeechGen: Unlocking the Generative Power of Speech Language Models with Promptscited by 21
  29. 2024Preference Alignment Improves Language Model-Based TTScited by 21
  30. 2024Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedbackcited by 19
  31. 2024ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Controlcited by 17
  32. 2024SimpleSpeech 2: Towards Simple and Efficient Text-to-Speech with Flow-based Scalar Latent Transformer Diffusion Modelscited by 17
  33. 2024Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decodingcited by 17
  34. 2024IntrinsicVoice: Empowering LLMs with Intrinsic Real-time Voice Interaction Abilitiescited by 16
  35. 2024CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversationscited by 15
  36. 2024IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTScited by 15
  37. 2024Paralinguistics-Aware Speech-Empowered Large Language Models for Natural Conversationcited by 14
  38. 2024FLEURS-R: A Restored Multilingual Speech Corpus for Generation Taskscited by 13

50 most cited of 167 All 167 (JSON)