# VoiceSculptor: Your Voice, Designed By You

arXiv: https://arxiv.org/abs/2601.10629 · HTML: https://mlsanity.com/paper/2601.10629/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2026.

## Built on by (most cited first, 10 of 10)

- [Qwen3-TTS Technical Report](https://mlsanity.com/paper/2601.15621/) (2026)
- [OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models](https://mlsanity.com/paper/2604.00688/) (2026)
- [VoXtream2: Full-stream TTS with dynamic speaking rate control](https://mlsanity.com/paper/2603.13518/) (2026)
- [MOSS-VoiceGenerator: Create Realistic Voices with Natural Language Descriptions](https://mlsanity.com/paper/2603.28086/) (2026)
- [FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generation](https://mlsanity.com/paper/2608.24168/) (2026)
- [AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing](https://mlsanity.com/paper/2609.08936/) (2026)
- [EmoInstruct-TTS: Dual-Path Instruction-Guided Emotional Speech Synthesis](https://mlsanity.com/paper/2606.20650/) (2026)
- [COT-TTS: Audio Context-Aware Text-to-Speech with Chain-of-Thought Reasoning](https://mlsanity.com/paper/2609.22697/) (2026)
- [JoyAI-Voice 2.0: A Full-Continuous Autoregressive Speech Generation Model with Semantic-Acoustic Joint Representation](https://mlsanity.com/paper/2610.08834/) (2026)
- [VoTSpeech: Decoupling Voice Design from Speech Generation in Instruction-Following TTS](https://mlsanity.com/paper/2610.14694/) (2026)

## Abstract (arXiv)

Despite rapid progress in text-to-speech (TTS), open-source systems still lack truly instruction-following, fine-grained control over core speech attributes (e.g., pitch, speaking rate, age, emotion, and style). We present VoiceSculptor, an open-source unified system that bridges this gap by integrating instruction-based voice design and high-fidelity voice cloning in a single framework. It generates controllable speaker timbre directly from natural-language descriptions, supports iterative refinement via Retrieval-Augmented Generation (RAG), and provides attribute-level edits across multiple dimensions. The designed voice is then rendered into a prompt waveform and fed into a cloning model to enable high-fidelity timbre transfer for downstream speech synthesis. VoiceSculptor achieves open-source state-of-the-art (SOTA) on InstructTTSEval-Zh, and is fully open-sourced, including code and pretrained models, to advance reproducible instruction-controlled TTS research.