mlsanityData to 14 Oct 2026

A Universal Music Translation Network

arXiv 1805.07848 ↗First posted 21 May 2018cs.SD cs.AI cs.LGcited by 22

Not on the map. Why?

Abstract (arXiv)

We present a method for translating music across musical instruments, genres, and styles. This method is based on a multi-domain wavenet autoencoder, with a shared encoder and a disentangled latent space that is trained end-to-end on waveforms. Employing a diverse training dataset and large net capacity, the domain-independent encoder allows us to translate even from musical domains that were not seen during training. The method is unsupervised and does not rely on supervision in the form of matched samples between domains or musical transcriptions. We evaluate our method on NSynth, as well as on a dataset collected from professional musicians, and achieve convincing translations, even when translating from whistling, potentially enabling the creation of instrumental music by untrained humans.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 May 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 6; 2023: 2; 2024: 3; 2025: 3; 2026: 1.

22 papers.

Sort:

By area:

  1. 2021RAVE: A variational autoencoder for fast and high-quality neural audio synthesiscited by 73
  2. 2022Multi-instrument Music Synthesis with Spectrogram Diffusioncited by 42
  3. 2021A Generative Model for Raw Audio Using Transformer Architecturescited by 18
  4. 2021Self-Supervised VQ-VAE for One-Shot Music Style Transfercited by 14
  5. 2024Music Style Transfer with Time-Varying Inversion of Diffusion Modelscited by 10
  6. 2024Combining audio control and style transfer using latent diffusioncited by 10
  7. 2022GANStrument: Adversarial Instrument Sound Synthesis with Pitch-invariant Instance Conditioningcited by 9
  8. 2021Timbre Transfer with Variational Auto Encoding and Cycle-Consistent Adversarial Networkscited by 8
  9. 2023Timbre transfer using image-to-image denoising diffusion implicit modelscited by 7
  10. 2022Disentangling speech from surroundings with neural embeddingscited by 6
  11. 2021CycleDRUMS: Automatic Drum Arrangement For Bass Lines Using CycleGANcited by 4
  12. 2021Discovery of Single Independent Latent Variablecited by 3
10 more
  1. 2022Leveraging Symmetrical Convolutional Transformer Networks for Speech to Singing Voice Style Transfercited by 2
  2. 2023Annotation-free Automatic Music Transcription with Scalable Synthetic Data and Adversarial Domain Confusioncited by 2
  3. 2025Latent Granular Resynthesis using Neural Audio Codecscited by 2
  4. 2021Musical Speech: A Transformer-based Composition Toolcited by 1
  5. 2022Conditional variational autoencoder to improve neural audio synthesis for polyphonic music soundcited by 1
  6. 2025Designing Neural Synthesizers for Low-Latency Interactioncited by 1
  7. 2025GuitarFlow: Realistic Electric Guitar Synthesis From Tablatures via Flow Matching and Style Transfercited by 1
  8. 2022Can Machines Generate Personalized Music? A Hybrid Favorite-aware Method for User Preference Music Transfercited by 0
  9. 2024Music Style Transfer With Diffusion Modelcited by 0
  10. 2026Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stemscited by 0