# TTSDS -- Text-to-Speech Distribution Score

arXiv: https://arxiv.org/abs/2407.12707 · HTML: https://mlsanity.com/paper/2407.12707/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 17 Jul 2024.

## Builds on (most cited first, 29 of 29)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models](https://mlsanity.com/paper/2403.03100/) (2024)
- [XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model](https://mlsanity.com/paper/2406.04904/) (2024)
- [StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models](https://mlsanity.com/paper/2306.07691/) (2023)
- [FastPitch: Parallel Text-to-speech with Pitch Prediction](https://mlsanity.com/paper/2006.06873/) (2020)
- [VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2406.05370/) (2024)
- [VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild](https://mlsanity.com/paper/2403.16973/) (2024)
- [LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpus](https://mlsanity.com/paper/2305.18802/) (2023)
- [Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit](https://mlsanity.com/paper/2210.17016/) (2022)
- [Natural language guidance of high-fidelity text-to-speech with synthetic annotations](https://mlsanity.com/paper/2402.01912/) (2024)
- [The VoiceMOS Challenge 2022](https://mlsanity.com/paper/2203.11389/) (2022)
- [VoiceFixer: Toward General Speech Restoration with Neural Vocoder](https://mlsanity.com/paper/2109.13731/) (2021)
- [A Spectral Energy Distance for Parallel Speech Synthesis](https://mlsanity.com/paper/2008.01160/) (2020)
- [Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model](https://mlsanity.com/paper/1804.01947/) (2018)
- [OpenVoice: Versatile Instant Voice Cloning](https://mlsanity.com/paper/2312.01479/) (2023)
- [A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speech](https://mlsanity.com/paper/2302.04215/) (2023)
- [Evaluating and reducing the distance between synthetic and real speech distributions](https://mlsanity.com/paper/2211.16049/) (2022)
- [Pheme: Efficient and Conversational Speech Generation](https://mlsanity.com/paper/2401.02839/) (2024)

## Built on by (most cited first, 14 of 14)

- [TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systems](https://mlsanity.com/paper/2506.19441/) (2025)
- [Exploiting Leaderboards for Large-Scale Distribution of Malicious Models](https://mlsanity.com/paper/2507.08983/) (2025)
- [Protect: Towards Robust Guardrailing Stack for Trustworthy Enterprise LLM Systems](https://mlsanity.com/paper/2510.13351/) (2025)
- [An Evaluation Framework for Text-to-Speech Voice Reconstruction](https://mlsanity.com/paper/2606.21343/) (2026)
- [Text-to-Image Models Leave Identifiable Signatures: Implications for Leaderboard Security](https://mlsanity.com/paper/2510.06525/) (2025)
- [Identifying Models Behind Text-to-Image Leaderboards](https://mlsanity.com/paper/2601.09647/) (2026)
- [Reliable Neural-Codec Text-to-Speech by ASR Self-Verification and Distillation: Near-Zero Catastrophic Failures Across Models and Codecs](https://mlsanity.com/paper/2606.18323/) (2026)
- [Is Natural Always Appropriate? Investigating Naturalness and Appropriateness Across Different Domains for TTS Evaluation](https://mlsanity.com/paper/2606.31729/) (2026)
- [CallScreenBench: Benchmarking Small Language Models as Phone Secretaries](https://mlsanity.com/paper/2608.01033/) (2026)
- [Iterative Self-Learning for Expressive Text-to-Speech Synthesis](https://mlsanity.com/paper/2608.15910/) (2026)
- [Complex-Text Robustness Evaluation and Failure Diagnosis for Low-Resource Multilingual Text-to-Speech](https://mlsanity.com/paper/2609.11545/) (2026)
- [Continuous-Time Acoustic Modelling with Neural Controlled Differential Equations](https://mlsanity.com/paper/2609.11725/) (2026)
- [Not Quite My Tempo: Voice Activity-aware Speech Synthesis for Lip-Synchronous Dubbing](https://mlsanity.com/paper/2609.26486/) (2026)
- [Distributional Metrics for Evaluating Spoken Conversational Systems](https://mlsanity.com/paper/2609.31719/) (2026)

## Abstract (arXiv)

Many recently published Text-to-Speech (TTS) systems produce audio close to real speech. However, TTS evaluation needs to be revisited to make sense of the results obtained with the new architectures, approaches and datasets. We propose evaluating the quality of synthetic speech as a combination of multiple factors such as prosody, speaker identity, and intelligibility. Our approach assesses how well synthetic speech mirrors real speech by obtaining correlates of each factor and measuring their distance from both real speech datasets and noise datasets. We benchmark 35 TTS systems developed between 2008 and 2024 and show that our score computed as an unweighted average of factors strongly correlates with the human evaluations from each time period.