# Fitting New Speakers Based on a Short Untranscribed Sample

arXiv: https://arxiv.org/abs/1802.06984 · HTML: https://mlsanity.com/paper/1802.06984/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2018.

## Built on by (most cited first, 18 of 18)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation](https://mlsanity.com/paper/2106.03153/) (2021)
- [Imaginary Voice: Face-styled Diffusion Model for Text-to-Speech](https://mlsanity.com/paper/2302.13700/) (2023)
- [V2C: Visual Voice Cloning](https://mlsanity.com/paper/2111.12890/) (2021)
- [Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Features](https://mlsanity.com/paper/2203.03191/) (2022)
- [AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Person](https://mlsanity.com/paper/2108.04325/) (2021)
- [CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editing](https://mlsanity.com/paper/2202.09950/) (2022)
- [Style-Label-Free: Cross-Speaker Style Transfer by Quantized VAE and Speaker-wise Normalization in Speech Synthesis](https://mlsanity.com/paper/2212.06397/) (2022)
- [Review of end-to-end speech synthesis technology based on deep learning](https://mlsanity.com/paper/2104.09995/) (2021)
- [Improve few-shot voice cloning using multi-modal learning](https://mlsanity.com/paper/2203.09708/) (2022)
- [Voice Cloning: a Multi-Speaker Text-to-Speech Synthesis Approach based on Transfer Learning](https://mlsanity.com/paper/2102.05630/) (2021)
- [The AS-NU System for the M2VoC Challenge](https://mlsanity.com/paper/2104.03009/) (2021)
- [Speaker Adaption with Intuitive Prosodic Features for Statistical Parametric Speech Synthesis](https://mlsanity.com/paper/2203.00951/) (2022)
- [AutoLV: Automatic Lecture Video Generator](https://mlsanity.com/paper/2209.08795/) (2022)
- [Controllable Generation of Artificial Speaker Embeddings through Discovery of Principal Directions](https://mlsanity.com/paper/2310.17502/) (2023)
- [Spiking Music: Audio Compression with Event Based Auto-encoders](https://mlsanity.com/paper/2402.01571/) (2024)
- [AI based Presentation Creator With Customized Audio Content Delivery](https://mlsanity.com/paper/2106.14213/) (2021)
- [Show Me Your Face, And I'll Tell You How You Speak](https://mlsanity.com/paper/2206.14009/) (2022)

## Abstract (arXiv)

Learning-based Text To Speech systems have the potential to generalize from one speaker to the next and thus require a relatively short sample of any new voice. However, this promise is currently largely unrealized. We present a method that is designed to capture a new speaker from a short untranscribed audio sample. This is done by employing an additional network that given an audio sample, places the speaker in the embedding space. This network is trained as part of the speech synthesis system using various consistency losses. Our results demonstrate a greatly improved performance on both the dataset speakers, and, more importantly, when fitting new voices, even from very short samples.