# A Universal Music Translation Network

arXiv: https://arxiv.org/abs/1805.07848 · HTML: https://mlsanity.com/paper/1805.07848/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2018.

## Built on by (most cited first, 22 of 22)

- [RAVE: A variational autoencoder for fast and high-quality neural audio synthesis](https://mlsanity.com/paper/2111.05011/) (2021)
- [Multi-instrument Music Synthesis with Spectrogram Diffusion](https://mlsanity.com/paper/2206.05408/) (2022)
- [A Generative Model for Raw Audio Using Transformer Architectures](https://mlsanity.com/paper/2106.16036/) (2021)
- [Self-Supervised VQ-VAE for One-Shot Music Style Transfer](https://mlsanity.com/paper/2102.05749/) (2021)
- [Music Style Transfer with Time-Varying Inversion of Diffusion Models](https://mlsanity.com/paper/2402.13763/) (2024)
- [Combining audio control and style transfer using latent diffusion](https://mlsanity.com/paper/2408.00196/) (2024)
- [GANStrument: Adversarial Instrument Sound Synthesis with Pitch-invariant Instance Conditioning](https://mlsanity.com/paper/2211.05385/) (2022)
- [Timbre Transfer with Variational Auto Encoding and Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/2109.02096/) (2021)
- [Timbre transfer using image-to-image denoising diffusion implicit models](https://mlsanity.com/paper/2307.04586/) (2023)
- [Disentangling speech from surroundings with neural embeddings](https://mlsanity.com/paper/2203.15578/) (2022)
- [CycleDRUMS: Automatic Drum Arrangement For Bass Lines Using CycleGAN](https://mlsanity.com/paper/2104.00353/) (2021)
- [Discovery of Single Independent Latent Variable](https://mlsanity.com/paper/2110.05887/) (2021)
- [Leveraging Symmetrical Convolutional Transformer Networks for Speech to Singing Voice Style Transfer](https://mlsanity.com/paper/2208.12410/) (2022)
- [Annotation-free Automatic Music Transcription with Scalable Synthetic Data and Adversarial Domain Confusion](https://mlsanity.com/paper/2312.10402/) (2023)
- [Latent Granular Resynthesis using Neural Audio Codecs](https://mlsanity.com/paper/2507.19202/) (2025)
- [Musical Speech: A Transformer-based Composition Tool](https://mlsanity.com/paper/2108.01043/) (2021)
- [Conditional variational autoencoder to improve neural audio synthesis for polyphonic music sound](https://mlsanity.com/paper/2211.08715/) (2022)
- [Designing Neural Synthesizers for Low-Latency Interaction](https://mlsanity.com/paper/2503.11562/) (2025)
- [GuitarFlow: Realistic Electric Guitar Synthesis From Tablatures via Flow Matching and Style Transfer](https://mlsanity.com/paper/2510.21872/) (2025)
- [Can Machines Generate Personalized Music? A Hybrid Favorite-aware Method for User Preference Music Transfer](https://mlsanity.com/paper/2201.08526/) (2022)
- [Music Style Transfer With Diffusion Model](https://mlsanity.com/paper/2404.14771/) (2024)
- [Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems](https://mlsanity.com/paper/2605.09259/) (2026)

## Abstract (arXiv)

We present a method for translating music across musical instruments, genres, and styles. This method is based on a multi-domain wavenet autoencoder, with a shared encoder and a disentangled latent space that is trained end-to-end on waveforms. Employing a diverse training dataset and large net capacity, the domain-independent encoder allows us to translate even from musical domains that were not seen during training. The method is unsupervised and does not rely on supervision in the form of matched samples between domains or musical transcriptions. We evaluate our method on NSynth, as well as on a dataset collected from professional musicians, and achieve convincing translations, even when translating from whistling, potentially enabling the creation of instrumental music by untrained humans.