# Context-Aware Prosody Correction for Text-Based Speech Editing

arXiv: https://arxiv.org/abs/2102.08328 · HTML: https://mlsanity.com/paper/2102.08328/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 16 Feb 2021.

## Builds on (most cited first, 10 of 10)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis](https://mlsanity.com/paper/1806.04558/) (2018)
- [FastPitch: Parallel Text-to-speech with Pitch Prediction](https://mlsanity.com/paper/2006.06873/) (2020)
- [CREPE: A Convolutional Representation for Pitch Estimation](https://mlsanity.com/paper/1802.06182/) (2018)
- [LPCNet: Improving Neural Speech Synthesis Through Linear Prediction](https://mlsanity.com/paper/1810.11846/) (2018)
- [HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks](https://mlsanity.com/paper/2006.05694/) (2020)
- [Controllable Neural Prosody Synthesis](https://mlsanity.com/paper/2008.03388/) (2020)

## Built on by (most cited first, 13 of 13)

- [VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild](https://mlsanity.com/paper/2403.16973/) (2024)
- [A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editing](https://mlsanity.com/paper/2203.09690/) (2022)
- [EdiTTS: Score-based Editing for Controllable Text-to-Speech](https://mlsanity.com/paper/2110.02584/) (2021)
- [FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Models](https://mlsanity.com/paper/2305.13612/) (2023)
- [CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editing](https://mlsanity.com/paper/2202.09950/) (2022)
- [Zero-Shot Text-to-Speech for Text-Based Insertion in Audio Narration](https://mlsanity.com/paper/2109.05426/) (2021)
- [Emotion Selectable End-to-End Text-based Speech Editing](https://mlsanity.com/paper/2212.10191/) (2022)
- [CorrectSpeech: A Fully Automated System for Speech Correction and Accent Reduction](https://mlsanity.com/paper/2204.05460/) (2022)
- [AttentionStitch: How Attention Solves the Speech Editing Problem](https://mlsanity.com/paper/2403.04804/) (2024)
- [Speech Editing -- a Summary](https://mlsanity.com/paper/2407.17172/) (2024)
- [Vox-Evaluator: Enhancing Stability and Fidelity for Zero-shot TTS with A Multi-Level Evaluator](https://mlsanity.com/paper/2510.20210/) (2025)
- [Towards zero-shot Text-based voice editing using acoustic context conditioning, utterance embeddings, and reference encoders](https://mlsanity.com/paper/2210.16045/) (2022)
- [Speak, Edit, Repeat: High-Fidelity Voice Editing and Zero-Shot TTS with Cross-Attentive Mamba](https://mlsanity.com/paper/2510.04738/) (2025)

## Abstract (arXiv)

Text-based speech editors expedite the process of editing speech recordings by permitting editing via intuitive cut, copy, and paste operations on a speech transcript. A major drawback of current systems, however, is that edited recordings often sound unnatural because of prosody mismatches around edited regions. In our work, we propose a new context-aware method for more natural sounding text-based editing of speech. To do so, we 1) use a series of neural networks to generate salient prosody features that are dependent on the prosody of speech surrounding the edit and amenable to fine-grained user control 2) use the generated features to control a standard pitch-shift and time-stretch method and 3) apply a denoising neural network to remove artifacts induced by the signal manipulation to yield a high-fidelity result. We evaluate our approach using a subjective listening test, provide a detailed comparative analysis, and conclude several interesting insights.