# Interpretable Style Transfer for Text-to-Speech with ControlVAE and Diffusion Bridge

arXiv: https://arxiv.org/abs/2306.04301 · HTML: https://mlsanity.com/paper/2306.04301/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2023.

## Built on by (most cited first, 3 of 3)

- [ReFlow-TTS: A Rectified Flow Model for High-fidelity Text-to-Speech](https://mlsanity.com/paper/2309.17056/) (2023)
- [VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editing](https://mlsanity.com/paper/2404.06674/) (2024)
- [SlimSpeech: Lightweight and Efficient Text-to-Speech with Slim Rectified Flow](https://mlsanity.com/paper/2504.07776/) (2025)

## Abstract (arXiv)

With the demand for autonomous control and personalized speech generation, the style control and transfer in Text-to-Speech (TTS) is becoming more and more important. In this paper, we propose a new TTS system that can perform style transfer with interpretability and high fidelity. Firstly, we design a TTS system that combines variational autoencoder (VAE) and diffusion refiner to get refined mel-spectrograms. Specifically, a two-stage and a one-stage system are designed respectively, to improve the audio quality and the performance of style transfer. Secondly, a diffusion bridge of quantized VAE is designed to efficiently learn complex discrete style representations and improve the performance of style transfer. To have a better ability of style transfer, we introduce ControlVAE to improve the reconstruction quality and have good interpretability simultaneously. Experiments on LibriTTS dataset demonstrate that our method is more effective than baseline models.