# Personalized Lightweight Text-to-Speech: Voice Cloning with Adaptive Structured Pruning

arXiv: https://arxiv.org/abs/2303.11816 · HTML: https://mlsanity.com/paper/2303.11816/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Mar 2023.

## Built on by (most cited first, 3 of 3)

- [Voice Cloning: Comprehensive Survey](https://mlsanity.com/paper/2505.00579/) (2025)
- [FMSD-TTS: Few-shot Multi-Speaker Multi-Dialect Text-to-Speech Synthesis for "U-Tsang, Amdo and Kham Speech Dataset Generation](https://mlsanity.com/paper/2505.14351/) (2025)
- [Fed-PISA: Federated Voice Cloning via Personalized Identity-Style Adaptation](https://mlsanity.com/paper/2509.16010/) (2025)

## Abstract (arXiv)

Personalized TTS is an exciting and highly desired application that allows users to train their TTS voice using only a few recordings. However, TTS training typically requires many hours of recording and a large model, making it unsuitable for deployment on mobile devices. To overcome this limitation, related works typically require fine-tuning a pre-trained TTS model to preserve its ability to generate high-quality audio samples while adapting to the target speaker's voice. This process is commonly referred to as ``voice cloning.'' Although related works have achieved significant success in changing the TTS model's voice, they are still required to fine-tune from a large pre-trained model, resulting in a significant size for the voice-cloned model. In this paper, we propose applying trainable structured pruning to voice cloning. By training the structured pruning masks with voice-cloning data, we can produce a unique pruned model for each target speaker. Our experiments demonstrate that using learnable structured pruning, we can compress the model size to 7 times smaller while achieving comparable voice-cloning performance.