# EditSpeech: A Text Based Speech Editing System Using Partial Inference and Bidirectional Fusion

arXiv: https://arxiv.org/abs/2107.01554 · HTML: https://mlsanity.com/paper/2107.01554/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jul 2021.

## Built on by (most cited first, 30 of 32)

- [VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild](https://mlsanity.com/paper/2403.16973/) (2024)
- [Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Bias](https://mlsanity.com/paper/2306.03509/) (2023)
- [A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing](https://mlsanity.com/paper/2203.09690/) (2022)
- [Autoregressive Diffusion Transformer for Text-to-Speech Synthesis](https://mlsanity.com/paper/2406.05551/) (2024)
- [EdiTTS: Score-based Editing for Controllable Text-to-Speech](https://mlsanity.com/paper/2110.02584/) (2021)
- [FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Models](https://mlsanity.com/paper/2305.13612/) (2023)
- [Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Survey](https://mlsanity.com/paper/2412.06602/) (2024)
- [SpeechPainter: Text-conditioned Speech Inpainting](https://mlsanity.com/paper/2202.07273/) (2022)
- [CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editing](https://mlsanity.com/paper/2202.09950/) (2022)
- [DiffVoice: Text-to-Speech with Latent Diffusion](https://mlsanity.com/paper/2304.11750/) (2023)
- [SongCreator: Lyrics-based Universal Song Generation](https://mlsanity.com/paper/2409.06029/) (2024)
- [uSee: Unified Speech Enhancement and Editing with Conditional Diffusion Models](https://mlsanity.com/paper/2310.00900/) (2023)
- [FluentEditor: Text-based Speech Editing by Considering Acoustic and Prosody Consistency](https://mlsanity.com/paper/2309.11725/) (2023)
- [Emotion Selectable End-to-End Text-based Speech Editing](https://mlsanity.com/paper/2212.10191/) (2022)
- [DiffEditor: Enhancing Speech Editing with Semantic Enrichment and Acoustic Consistency](https://mlsanity.com/paper/2409.12992/) (2024)
- [SeniorTalk: A Chinese Conversation Dataset with Rich Annotations for Super-Aged Seniors](https://mlsanity.com/paper/2503.16578/) (2025)
- [Improving Code-Switching and Named Entity Recognition in ASR with Speech Editing based Data Augmentation](https://mlsanity.com/paper/2306.08588/) (2023)
- [CorrectSpeech: A Fully Automated System for Speech Correction and Accent Reduction](https://mlsanity.com/paper/2204.05460/) (2022)
- [Cross-Utterance Conditioned VAE for Speech Generation](https://mlsanity.com/paper/2309.04156/) (2023)
- [AttentionStitch: How Attention Solves the Speech Editing Problem](https://mlsanity.com/paper/2403.04804/) (2024)
- [Speech Editing -- a Summary](https://mlsanity.com/paper/2407.17172/) (2024)
- [FluentEditor2: Text-based Speech Editing by Modeling Multi-Scale Acoustic and Prosody Consistency](https://mlsanity.com/paper/2410.03719/) (2024)
- [Towards zero-shot Text-based voice editing using acoustic context conditioning, utterance embeddings, and reference encoders](https://mlsanity.com/paper/2210.16045/) (2022)
- [ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech](https://mlsanity.com/paper/2211.03545/) (2022)
- [Automatic Voice Identification after Speech Resynthesis using PPG](https://mlsanity.com/paper/2408.02712/) (2024)
- [Towards Emotionally Consistent Text-Based Speech Editing: Introducing EmoCorrector and The ECD-TSE Dataset](https://mlsanity.com/paper/2505.20341/) (2025)
- [Speak, Edit, Repeat: High-Fidelity Voice Editing and Zero-Shot TTS with Cross-Attentive Mamba](https://mlsanity.com/paper/2510.04738/) (2025)
- [AST: Adaptive, Seamless, and Training-Free Precise Speech Editing](https://mlsanity.com/paper/2604.16056/) (2026)
- [dots.tts.edit: Precisely Controlled Speech Editing with a Continuous Autoregressive Model](https://mlsanity.com/paper/2608.02673/) (2026)
- [Multi Codec Discrete Diffusion Model for Text Guided Speech Inpainting and Editing](https://mlsanity.com/paper/2608.06424/) (2026)

## Abstract (arXiv)

This paper presents the design, implementation and evaluation of a speech editing system, named EditSpeech, which allows a user to perform deletion, insertion and replacement of words in a given speech utterance, without causing audible degradation in speech quality and naturalness. The EditSpeech system is developed upon a neural text-to-speech (NTTS) synthesis framework. Partial inference and bidirectional fusion are proposed to effectively incorporate the contextual information related to the edited region and achieve smooth transition at both left and right boundaries. Distortion introduced to the unmodified parts of the utterance is alleviated. The EditSpeech system is developed and evaluated on English and Chinese in multi-speaker scenarios. Objective and subjective evaluation demonstrate that EditSpeech outperforms a few baseline systems in terms of low spectral distortion and preferred speech quality. Audio samples are available online for demonstration https://daxintan-cuhk.github.io/EditSpeech/ .