dots.tts.edit: Precisely Controlled Speech Editing with a Continuous Autoregressive Model
Not on the map. Why?
Abstract (arXiv)
Speech editing for content creation requires precise control over both what an edit should do and where it should apply. Free-form natural language provides a flexible interface for expressing edit requests, but its ambiguity may leave the intended operation, parameters, or target region underspecified. We study a precise and explicit interface for speech editing: a transcript-grounded structural edit instruction with XML-style tags explicitly specifies typed operations and localizes them to transcript spans or boundaries. This semantic timeline avoids explicit timestamp alignment and provides an externally inspectable contract for compositional edits. We instantiate the interface in dots.tts.edit, an editor adapted from the continuous autoregressive dots.tts foundation model. Four representative speech-creation controls cover lexical content, affective expression, pitch and speaking-rate delivery, and temporal phrasing through text, emotion, prosody, and pause editing. Task-specific data pipelines construct operation- and scope-controlled pairs while retaining source-derived context outside each target region. We further introduce doteBench, a bilingual evaluation suite that measures precise instruction following, local preservation, and audio quality across the four controls and their composition. Experiments show leading overall instruction following and local preservation across its five editing categories, while audio quality remains comparable to existing open-source systems. Across three Seed-TTS-Eval shards, the model shows negligible differences from the base model in zero-shot TTS recognition error rate and speaker similarity.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 45 references that are in the map. First posted on arXiv 2 Aug 2026.
Builds on
30 of 45 references.
References · 30
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 723
- 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
- 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
- 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
- 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 120
- 2023CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Maskingcited by 102
- 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 90
18 more
- 2025MiMo-Audio: Audio Language Models are Few-Shot Learnerscited by 65
- 2023SpeechX: Neural Codec Language Model as a Versatile Speech Transformercited by 64
- 2022A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editingcited by 40
- 2023UniCATS: A Unified Context-Aware Text-to-Speech Framework with Contextual VQ-Diffusion and Vocodingcited by 37
- 2021EditSpeech: A Text Based Speech Editing System Using Partial Inference and Bidirectional Fusioncited by 32
- 2022CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editingcited by 16
- 2026Audio-Omni: Extending Multi-modal Understanding to Versatile Audio Generation and Editingcited by 12
- 2025MMEDIT: A Unified Framework for Multi-Type Audio Editing via Audio Language Modelcited by 9
- 2023FluentEditor: Text-based Speech Editing by Considering Acoustic and Prosody Consistencycited by 6
- 2026MMAE: A Massive Multitask Audio Editing Benchmarkcited by 5
- 2026CosyEdit: Unlocking End-to-End Speech Editing Capability from Zero-Shot Text-to-Speech Modelscited by 4
- 2026SpeechEditBench: A Bilingual Multi-Attribute Benchmark for Instruction-Guided Speech Editingcited by 4
- 2025ISSE: An Instruction-Guided Speech Style Editing Dataset And Benchmarkcited by 3
- 2026A Unified Neural Codec Language Model for Selective Editable Text to Speech Generationcited by 2
- 2026HoliTok: A Continuous Holistic Tokenization with Robust Dual Capabilities of Speech Generation and Understandingcited by 2
- 2026UNISON: A Unified Sound Generation and Editing Framework via Deep LLM Fusioncited by 2
- 2024FluentEditor2: Text-based Speech Editing by Modeling Multi-Scale Acoustic and Prosody Consistencycited by 1
- 2026TED-TTS: Training-Free Intra-Utterance Emotion and Duration Control for Text-to-Speech Synthesiscited by 1
Built on by
Not cited yet.