# A Unified Neural Codec Language Model for Selective Editable Text to Speech Generation

arXiv: https://arxiv.org/abs/2601.12480 · HTML: https://mlsanity.com/paper/2601.12480/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jan 2026.

## Built on by (most cited first, 2 of 2)

- [dots.tts.edit: Precisely Controlled Speech Editing with a Continuous Autoregressive Model](https://mlsanity.com/paper/2608.02673/) (2026)
- [SEmoEdit: Probing and Harnessing the Editability of Pre-trained Speech Flows](https://mlsanity.com/paper/2609.34648/) (2026)

## Abstract (arXiv)

Neural codec language models achieve impressive zero-shot Text-to-Speech (TTS) by fully imitating the acoustic characteristics of a short speech prompt, including timbre, prosody, and paralinguistic information. However, such holistic imitation limits their ability to isolate and control individual attributes. In this paper, we present a unified codec language model SpeechEdit that extends zero-shot TTS with a selective control mechanism. By default, SpeechEdit reproduces the complete acoustic profile inferred from the speech prompt, but it selectively overrides only the attributes specified by explicit control instructions. To enable controllable modeling, SpeechEdit is trained on our newly constructed LibriEdit dataset, which provides delta (difference-aware) training pairs derived from LibriHeavy. Experimental results show that our approach maintains naturalness and robustness while offering flexible and localized control over desired attributes. Audio samples are available at https://speech-editing.github.io/speech-editing/.