mlsanityData to 7 Oct 2026

EdiTTS: Score-based Editing for Controllable Text-to-Speech

arXiv 2110.02584 ↗First posted 6 Oct 2021cs.SD cs.LG eess.AScited by 28

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

We present EdiTTS, an off-the-shelf speech editing methodology based on score-based generative modeling for text-to-speech synthesis. EdiTTS allows for targeted, granular editing of audio, both in terms of content and pitch, without the need for any additional training, task-specific optimization, or architectural modifications to the score-based model backbone. Specifically, we apply coarse yet deliberate perturbations in the Gaussian prior space to induce desired behavior from the diffusion model while applying masks and softening kernels to ensure that iterative edits are applied only to the target region. Through listening tests and speech-to-text back transcription, we show that EdiTTS outperforms existing baselines and produces robust samples that satisfy user-imposed requirements.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 18 of its 26 references that are in the map. First posted on arXiv 6 Oct 2021.

Builds on

18 of 26 references.

Closest ancestors · 4

Same direction, newest first

  1. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme28 Sep
  2. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech13 May
  3. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speech3 Apr
  4. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 14

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  7. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  8. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  9. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  10. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
  11. 2018CREPE: A Convolutional Representation for Pitch Estimationcited by 109
  12. 2021One TTS Alignment To Rule Them Allcited by 47
2 more
  1. 2021EditSpeech: A Text Based Speech Editing System Using Partial Inference and Bidirectional Fusioncited by 31
  2. 2021Context-Aware Prosody Correction for Text-Based Speech Editingcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 10; 2024: 8; 2025: 3; 2026: 2.

28 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  2. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  3. 2022A Survey on Generative Diffusion Modelcited by 238
  4. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matchingcited by 49
  5. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesiscited by 34
  6. 2023E3 TTS: Easy End-to-End Diffusion-based Text to Speechcited by 30
  7. 2025TESS 2: A Large-Scale Generalist Diffusion Language Modelcited by 30
  8. 2023Speech Driven Video Editing via an Audio-Conditioned Diffusion Modelcited by 28
  9. 2023Diffusion-based Image Translation with Label Guidance for Domain Adaptive Semantic Segmentationcited by 27
  10. 2023FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Modelscited by 23
  11. 2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Surveycited by 22
  12. 2023DiffTAD: Temporal Action Detection with Proposal Denoising Diffusioncited by 19
16 more
  1. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
  2. 2025Ming-UniAudio: Speech LLM for Joint Understanding, Generation and Editing with Unified Representationcited by 15
  3. 2023DiffusEmp: A Diffusion Model-Based Framework with Multi-Grained Control for Empathetic Response Generationcited by 9
  4. 2023uSee: Unified Speech Enhancement and Editing with Conditional Diffusion Modelscited by 7
  5. 2024DLPO: Diffusion Model Loss-Guided Reinforcement Learning for Fine-Tuning Text-to-Speech Diffusion Modelscited by 7
  6. 2023FluentEditor: Text-based Speech Editing by Considering Acoustic and Prosody Consistencycited by 6
  7. 2024Gated Temporal Diffusion for Stochastic Long-Term Dense Anticipationcited by 6
  8. 2022CorrectSpeech: A Fully Automated System for Speech Correction and Accent Reductioncited by 1
  9. 2024AttentionStitch: How Attention Solves the Speech Editing Problemcited by 1
  10. 2024Speech Editing -- a Summarycited by 1
  11. 2024FluentEditor2: Text-based Speech Editing by Modeling Multi-Scale Acoustic and Prosody Consistencycited by 1
  12. 2025SupertonicTTS: Towards Highly Efficient and Streamlined Text-to-Speech Systemcited by 1
  13. 2026Leveraging Pretrained Language Models as Energy Functions for Glauber Dynamics Text Diffusioncited by 1
  14. 2026UNISON: A Unified Sound Generation and Editing Framework via Deep LLM Fusioncited by 1
  15. 2022Towards zero-shot Text-based voice editing using acoustic context conditioning, utterance embeddings, and reference encoderscited by 0
  16. 2024Automatic Voice Identification after Speech Resynthesis using PPGcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SAO-Instruct: Free-form Audio Editing using Natural Language Instructions
  2. 2026Edit Content, Preserve Acoustics: Imperceptible Text-Based Speech Editing via Self-Consistency Rewards ↗
  3. 2025Speak, Edit, Repeat: High-Fidelity Voice Editing and Zero-Shot TTS with Cross-Attentive Mamba
  4. 2026AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing
  5. 2025Audio Palette: A Diffusion Transformer with Multi-Signal Conditioning for Controllable Foley Synthesis
  6. 2024On the Semantic Latent Space of Diffusion-Based Text-to-Speech Models
  7. 2026LLaDA-TTS: Unifying Speech Synthesis and Zero-Shot Editing via Masked Diffusion Modeling
  8. 2026SongCraft: Unified Song Generation and Editing with Reconstructive Learning ↗