mlsanityData to 7 Oct 2026

SpeechPainter: Text-conditioned Speech Inpainting

arXiv 2202.07273 ↗First posted 15 Feb 2022cs.SD cs.LG eess.AScited by 17

Not on the map. Why?

Abstract (arXiv)

We propose SpeechPainter, a model for filling in gaps of up to one second in speech samples by leveraging an auxiliary textual input. We demonstrate that the model performs speech inpainting with the appropriate content, while maintaining speaker identity, prosody and recording environment conditions, and generalizing to unseen speakers. Our approach significantly outperforms baselines constructed using adaptive TTS, as judged by human raters in side-by-side preference and MOS tests.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 18 of its 27 references that are in the map. First posted on arXiv 15 Feb 2022.

Builds on

18 of 27 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Layer Normalizationcited by 5,579
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  5. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
  6. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  7. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  8. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
  9. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  10. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  11. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 171
  12. 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
6 more
  1. 2021Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generationcited by 93
  2. 2018VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Maskingcited by 72
  3. 2017Audio-Visual Speech Enhancement Using Multimodal Deep Convolutional Neural Networkscited by 36
  4. 2020SEANet: A Multi-modal Speech Enhancement Networkcited by 35
  5. 2021EditSpeech: A Text Based Speech Editing System Using Partial Inference and Bidirectional Fusioncited by 31
  6. 2020Audio-Visual Speech Inpainting with Deep Learningcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 4; 2025: 5; 2026: 1.

17 papers.

Sort:

By area:

  1. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  2. 2023AUDIT: Audio Editing by Following Instructions with Latent Diffusion Modelscited by 81
  3. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesiscited by 34
  4. 2023Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representationscited by 22
  5. 2023DiffVoice: Text-to-Speech with Latent Diffusioncited by 13
  6. 2024SongCreator: Lyrics-based Universal Song Generationcited by 12
  7. 2025A2SB: Audio-to-Audio Schrodinger Bridgescited by 11
  8. 2023Speech inpainting: Context-based speech synthesis guided by videocited by 4
  9. 2025VoiceCraft-X: Unifying Multilingual, Voice-Cloning Speech Synthesis and Speech Editingcited by 3
  10. 2023Improving Code-Switching and Named Entity Recognition in ASR with Speech Editing based Data Augmentationcited by 2
  11. 2023Controllable Prosody Generation With Partial Inputscited by 1
  12. 2024AttentionStitch: How Attention Solves the Speech Editing Problemcited by 1
5 more
  1. 2024Robust Multi-Modal Speech In-Painting: A Sequence-to-Sequence Approachcited by 0
  2. 2025Text-Queried Audio Source Separation via Hierarchical Modelingcited by 0
  3. 2025BNMusic: Blending Environmental Noises into Personalized Musiccited by 0
  4. 2025Speak, Edit, Repeat: High-Fidelity Voice Editing and Zero-Shot TTS with Cross-Attentive Mambacited by 0
  5. 2026Multi Codec Discrete Diffusion Model for Text Guided Speech Inpainting and Editingcited by 0