# On Using Backpropagation for Speech Texture Generation and Voice Conversion

arXiv: https://arxiv.org/abs/1712.08363 · HTML: https://mlsanity.com/paper/1712.08363/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Dec 2017.

## Built on by (most cited first, 2 of 2)

- [Protecting gender and identity with disentangled speech representations](https://mlsanity.com/paper/2104.11051/) (2021)
- [Speaker embeddings by modeling channel-wise correlations](https://mlsanity.com/paper/2104.02571/) (2021)

## Abstract (arXiv)

Inspired by recent work on neural network image generation which rely on backpropagation towards the network inputs, we present a proof-of-concept system for speech texture synthesis and voice conversion based on two mechanisms: approximate inversion of the representation learned by a speech recognition neural network, and on matching statistics of neuron activations between different source and target utterances. Similar to image texture synthesis and neural style transfer, the system works by optimizing a cost function with respect to the input waveform samples. To this end we use a differentiable mel-filterbank feature extraction pipeline and train a convolutional CTC speech recognition network. Our system is able to extract speaker characteristics from very limited amounts of target speaker data, as little as a few seconds, and can be used to generate realistic speech babble or reconstruct an utterance in a different voice.