# RephraseTTS: Dynamic Length Text based Speech Insertion with Speaker Style Transfer

arXiv: https://arxiv.org/abs/2508.17031 · HTML: https://mlsanity.com/paper/2508.17031/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 18 references that are in the map.
- First posted on arXiv 23 Aug 2025.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis](https://mlsanity.com/paper/1806.04558/) (2018)
- [Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation](https://mlsanity.com/paper/2106.03153/) (2021)
- [SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Model](https://mlsanity.com/paper/2104.05557/) (2021)
- [Sample Efficient Adaptive Text-to-Speech](https://mlsanity.com/paper/1809.10460/) (2018)
- [Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embedding](https://mlsanity.com/paper/2005.08484/) (2020)
- [RetrieverTTS: Modeling Decomposed Factors for Text-Based Speech Insertion](https://mlsanity.com/paper/2206.13865/) (2022)
- [Zero-Shot Text-to-Speech for Text-Based Insertion in Audio Narration](https://mlsanity.com/paper/2109.05426/) (2021)

## Abstract (arXiv)

We propose a method for the task of text-conditioned speech insertion, i.e. inserting a speech sample in an input speech sample, conditioned on the corresponding complete text transcript. An example use case of the task would be to update the speech audio when corrections are done on the corresponding text transcript. The proposed method follows a transformer-based non-autoregressive approach that allows speech insertions of variable lengths, which are dynamically determined during inference, based on the text transcript and tempo of the available partial input. It is capable of maintaining the speaker's voice characteristics, prosody and other spectral properties of the available speech input. Results from our experiments and user study on LibriTTS show that our method outperforms baselines based on an existing adaptive text to speech method. We also provide numerous qualitative results to appreciate the quality of the output from the proposed method.