# CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editing

arXiv: https://arxiv.org/abs/2202.09950 · HTML: https://mlsanity.com/paper/2202.09950/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 55 references that are in the map.
- First posted on arXiv 21 Feb 2022.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis](https://mlsanity.com/paper/1806.04558/) (2018)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [Efficient Neural Audio Synthesis](https://mlsanity.com/paper/1802.08435/) (2018)
- [An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning](https://mlsanity.com/paper/2008.03648/) (2020)
- [LPCNet: Improving Neural Speech Synthesis Through Linear Prediction](https://mlsanity.com/paper/1810.11846/) (2018)
- [HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks](https://mlsanity.com/paper/2006.05694/) (2020)
- [EditSpeech: A Text Based Speech Editing System Using Partial Inference and Bidirectional Fusion](https://mlsanity.com/paper/2107.01554/) (2021)
- [Fitting New Speakers Based on a Short Untranscribed Sample](https://mlsanity.com/paper/1802.06984/) (2018)
- [Context-Aware Prosody Correction for Text-Based Speech Editing](https://mlsanity.com/paper/2102.08328/) (2021)
- [A Neural Vocoder with Hierarchical Generation of Amplitude and Phase Spectra for Statistical Parametric Speech Synthesis](https://mlsanity.com/paper/1906.09573/) (2019)
- [End-to-End Attention based Text-Dependent Speaker Verification](https://mlsanity.com/paper/1701.00562/) (2017)

## Built on by (most cited first, 16 of 16)

- [NExT-GPT: Any-to-Any Multimodal LLM](https://mlsanity.com/paper/2309.05519/) (2023)
- [DiffVoice: Text-to-Speech with Latent Diffusion](https://mlsanity.com/paper/2304.11750/) (2023)
- [SongCreator: Lyrics-based Universal Song Generation](https://mlsanity.com/paper/2409.06029/) (2024)
- [FluentEditor: Text-based Speech Editing by Considering Acoustic and Prosody Consistency](https://mlsanity.com/paper/2309.11725/) (2023)
- [Spider: Any-to-Many Multimodal LLM](https://mlsanity.com/paper/2411.09439/) (2024)
- [Emotion Selectable End-to-End Text-based Speech Editing](https://mlsanity.com/paper/2212.10191/) (2022)
- [DiffEditor: Enhancing Speech Editing with Semantic Enrichment and Acoustic Consistency](https://mlsanity.com/paper/2409.12992/) (2024)
- [SeniorTalk: A Chinese Conversation Dataset with Rich Annotations for Super-Aged Seniors](https://mlsanity.com/paper/2503.16578/) (2025)
- [CorrectSpeech: A Fully Automated System for Speech Correction and Accent Reduction](https://mlsanity.com/paper/2204.05460/) (2022)
- [Cross-Utterance Conditioned VAE for Speech Generation](https://mlsanity.com/paper/2309.04156/) (2023)
- [FluentEditor2: Text-based Speech Editing by Modeling Multi-Scale Acoustic and Prosody Consistency](https://mlsanity.com/paper/2410.03719/) (2024)
- [Towards zero-shot Text-based voice editing using acoustic context conditioning, utterance embeddings, and reference encoders](https://mlsanity.com/paper/2210.16045/) (2022)
- [AST: Adaptive, Seamless, and Training-Free Precise Speech Editing](https://mlsanity.com/paper/2604.16056/) (2026)
- [dots.tts.edit: Precisely Controlled Speech Editing with a Continuous Autoregressive Model](https://mlsanity.com/paper/2608.02673/) (2026)
- [Multi Codec Discrete Diffusion Model for Text Guided Speech Inpainting and Editing](https://mlsanity.com/paper/2608.06424/) (2026)
- [AegisVox: Factorized Privacy and Clinical Preservation for Long-Form Speech](https://mlsanity.com/paper/2610.14643/) (2026)

## Abstract (arXiv)

The text-based speech editor allows the editing of speech through intuitive cutting, copying, and pasting operations to speed up the process of editing speech. However, the major drawback of current systems is that edited speech often sounds unnatural due to cut-copy-paste operation. In addition, it is not obvious how to synthesize records according to a new word not appearing in the transcript. This paper proposes a novel end-to-end text-based speech editing method called context-aware mask prediction network (CampNet). The model can simulate the text-based speech editing process by randomly masking part of speech and then predicting the masked region by sensing the speech context. It can solve unnatural prosody in the edited region and synthesize the speech corresponding to the unseen words in the transcript. Secondly, for the possible operation of text-based speech editing, we design three text-based operations based on CampNet: deletion, insertion, and replacement. These operations can cover various situations of speech editing. Thirdly, to synthesize the speech corresponding to long text in insertion and replacement operations, a word-level autoregressive generation method is proposed. Fourthly, we propose a speaker adaptation method using only one sentence for CampNet and explore the ability of few-shot learning based on CampNet, which provides a new idea for speech forgery tasks. The subjective and objective experiments on VCTK and LibriTTS datasets show that the speech editing results based on CampNet are better than TTS technology, manual editing, and VoCo method. We also conduct detailed ablation experiments to explore the effect of the CampNet structure on its performance. Finally, the experiment shows that speaker adaptation with only one sentence can further improve the naturalness of speech. Examples of generated speech can be found at https://hairuo55.github.io/CampNet.