Speech Editing -- a Summary
Not on the map. Why?
Abstract (arXiv)
With the rise of video production and social media, speech editing has become crucial for creators to address issues like mispronunciations, missing words, or stuttering in audio recordings. This paper explores text-based speech editing methods that modify audio via text transcripts without manual waveform editing. These approaches ensure edited audio is indistinguishable from the original by altering the mel-spectrogram. Recent advancements, such as context-aware prosody correction and advanced attention mechanisms, have improved speech editing quality. This paper reviews state-of-the-art methods, compares key metrics, and examines widely used datasets. The aim is to highlight ongoing issues and inspire further research and innovation in speech editing.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 28 of its 48 references that are in the map. First posted on arXiv 24 Jul 2024.
Builds on
28 of 48 references.
References · 28
- 2017Attention Is All You Needcited by 59,932
- 2017Neural Discrete Representation Learningcited by 4,650
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
- 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
- 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
16 more
- 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
- 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
- 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
- 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 235
- 2021DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressorscited by 165
- 2019MOSNet: Deep Learning based Objective Assessment for Voice Conversioncited by 126
- 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
- 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 118
- 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 71
- 2023SpeechX: Neural Codec Language Model as a Versatile Speech Transformercited by 64
- 2022A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editingcited by 40
- 2021EditSpeech: A Text Based Speech Editing System Using Partial Inference and Bidirectional Fusioncited by 31
- 2021EdiTTS: Score-based Editing for Controllable Text-to-Speechcited by 28
- 2023FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Modelscited by 23
- 2021Context-Aware Prosody Correction for Text-Based Speech Editingcited by 13
- 2022RetrieverTTS: Modeling Decomposed Factors for Text-Based Speech Insertioncited by 10
Built on by
Citing papers per year: 2026: 1.
1 paper.