# Latent Diffusion Bridges for Unsupervised Musical Audio Timbre Transfer

arXiv: https://arxiv.org/abs/2409.06096 · HTML: https://mlsanity.com/paper/2409.06096/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 32 of its 38 references that are in the map.
- First posted on arXiv 9 Sep 2024.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 30 of 32)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling](https://mlsanity.com/paper/2106.01357/) (2021)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [Dual Diffusion Implicit Bridges for Image-to-Image Translation](https://mlsanity.com/paper/2203.08382/) (2022)
- [Denoising Diffusion Bridge Models](https://mlsanity.com/paper/2309.16948/) (2023)
- [Adapting Frechet Audio Distance for Generative Music Evaluation](https://mlsanity.com/paper/2311.01616/) (2023)
- [StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks](https://mlsanity.com/paper/1806.02169/) (2018)
- [Multi-Source Diffusion Models for Simultaneous Music Generation and Separation](https://mlsanity.com/paper/2302.02257/) (2023)
- [Solving Audio Inverse Problems with a Diffusion Model](https://mlsanity.com/paper/2210.15228/) (2022)
- [Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion](https://mlsanity.com/paper/2402.10009/) (2024)
- [Convergence Analysis of Probability Flow ODE for Score-based Generative Models](https://mlsanity.com/paper/2404.09730/) (2024)
- [TimbreTron: A WaveNet(CycleGAN(CQT(Audio))) Pipeline for Musical Timbre Transfer](https://mlsanity.com/paper/1811.09620/) (2018)
- [MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models](https://mlsanity.com/paper/2402.06178/) (2024)
- [A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and Multipitch Estimation](https://mlsanity.com/paper/2203.09893/) (2022)
- [FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Models](https://mlsanity.com/paper/2401.15636/) (2024)
- [Arrange, Inpaint, and Refine: Steerable Long-term Music Audio Generation and Editing via Content-based Controls](https://mlsanity.com/paper/2402.09508/) (2024)
- [Instruct-MusicGen: Unlocking Text-to-Music Editing for Music Language Models via Instruction Tuning](https://mlsanity.com/paper/2405.18386/) (2024)
- [Self-Supervised VQ-VAE for One-Shot Music Style Transfer](https://mlsanity.com/paper/2102.05749/) (2021)
- [COCOLA: Coherence-Oriented Contrastive Learning of Musical Audio Representations](https://mlsanity.com/paper/2404.16969/) (2024)
- [The Chamber Ensemble Generator: Limitless High-Quality MIR Data via Generative Modeling](https://mlsanity.com/paper/2209.14458/) (2022)
- [Modulated Variational auto-Encoders for many-to-many musical timbre transfer](https://mlsanity.com/paper/1810.00222/) (2018)
- [Timbre Transfer with Variational Auto Encoding and Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/2109.02096/) (2021)
- [Timbre transfer using image-to-image denoising diffusion implicit models](https://mlsanity.com/paper/2307.04586/) (2023)
- [Vector-Quantized Timbre Representation](https://mlsanity.com/paper/2007.06349/) (2020)
- [Towards Robust Unsupervised Disentanglement of Sequential Data -- A Case Study Using Music Audio](https://mlsanity.com/paper/2205.05871/) (2022)
- [Gaussian Flow Bridges for Audio Domain Transfer with Unpaired Data](https://mlsanity.com/paper/2405.19497/) (2024)

## Built on by (most cited first, 9 of 9)

- [Diffusion Timbre Transfer Via Mutual Information Guided Inpainting](https://mlsanity.com/paper/2601.01294/) (2026)
- [Towards General Modality Translation with Contrastive and Predictive Latent Diffusion Bridge](https://mlsanity.com/paper/2510.20819/) (2025)
- [DARC: Drum accompaniment generation with fine-grained rhythm control](https://mlsanity.com/paper/2601.02357/) (2026)
- [Designing Neural Synthesizers for Low-Latency Interaction](https://mlsanity.com/paper/2503.11562/) (2025)
- [Annotation-Free MIDI-to-Audio Synthesis via Concatenative Synthesis and Generative Refinement](https://mlsanity.com/paper/2410.16785/) (2024)
- [Diffusion-based translation between unpaired spontaneous premature neonatal EEG and fetal MEG](https://mlsanity.com/paper/2507.14224/) (2025)
- [Remix the Timbre: Diffusion-Based Style Transfer Across Polyphonic Stems](https://mlsanity.com/paper/2605.09259/) (2026)
- [PianoKontext: Expressive Performance Rendering from Deadpan Context](https://mlsanity.com/paper/2606.12282/) (2026)
- [Differentiable Digital Signal Processing Mixture Model-Guided Diffusion for Synthesis Parameter Estimation from Harmonic Sound Mixtures](https://mlsanity.com/paper/2609.14427/) (2026)

## Abstract (arXiv)

Music timbre transfer is a challenging task that involves modifying the timbral characteristics of an audio signal while preserving its melodic structure. In this paper, we propose a novel method based on dual diffusion bridges, trained using the CocoChorales Dataset, which consists of unpaired monophonic single-instrument audio data. Each diffusion model is trained on a specific instrument with a Gaussian prior. During inference, a model is designated as the source model to map the input audio to its corresponding Gaussian prior, and another model is designated as the target model to reconstruct the target audio from this Gaussian prior, thereby facilitating timbre transfer. We compare our approach against existing unsupervised timbre transfer models such as VAEGAN and Gaussian Flow Bridges (GFB). Experimental results demonstrate that our method achieves both better Fr'echet Audio Distance (FAD) and melody preservation, as reflected by lower pitch distances (DPD) compared to VAEGAN and GFB. Additionally, we discover that the noise level from the Gaussian prior, σ, can be adjusted to control the degree of melody preservation and amount of timbre transferred.