Differentiable Digital Signal Processing Mixture Model-Guided Diffusion for Synthesis Parameter Estimation from Harmonic Sound Mixtures
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
A differentiable digital signal processing (DDSP) autoencoder reconstructs a monophonic harmonic sound through three types of synthesis parameters: fundamental frequency, loudness, and timbre features. To handle mixtures of harmonic sounds within the DDSP approach, we have previously proposed a DDSP mixture model (DDSPMM). It represents a mixture as the sum of source signals synthesized by the decoders of pretrained DDSP autoencoders. Although DDSPMM enables direct estimation of synthesis parameters of each source from mixtures, it does not explicitly model temporal variations in the synthesis parameters and can produce excessive temporal fluctuations. In this paper, we propose a method for estimating synthesis parameters with temporally plausible trajectories by incorporating a denoising diffusion probabilistic model (DDPM) into the DDSPMM-based estimation. The DDPM is trained as a generative model of synthesis parameters. During estimation, the proposed method guides the DDPM reverse diffusion process with the reconstruction error between the observed mixture and the mixture synthesized by DDSPMM from the current estimates. Experiments on woodwind and string instrument ensembles showed that the DDPM-based regularization improves synthesis parameter estimation by imposing temporal plausibility on the estimated trajectories.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 15 of its 17 references that are in the map. First posted on arXiv 13 Sep 2026.
Builds on
15 of 17 references.
Closest ancestors · 2
Same direction, newest first
- 2024Latent Diffusion Bridges for Unsupervised Musical Audio Timbre Transfer9 Sep
- 2021CRASH: Raw Audio Score-based Generative Modeling for Controllable High-resolution Drum Sound Synthesis14 Jun
Other references · 13
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
- 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 225
- 2020DDSP: Differentiable Digital Signal Processingcited by 178
- 2018CREPE: A Convolutional Representation for Pitch Estimationcited by 110
- 2021RAVE: A variational autoencoder for fast and high-quality neural audio synthesiscited by 73
- 2021MIDI-DDSP: Detailed Control of Musical Performance via Hierarchical Modelingcited by 32
- 2023A Review of Differentiable Digital Signal Processing for Music & Speech Synthesiscited by 20
- 2025Zimtohrli: An Efficient Psychoacoustic Audio Similarity Metriccited by 6
- 2024Expressive Acoustic Guitar Sound Synthesis with an Instrument-Specific Input Representation and Diffusion Outpaintingcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Timbre transfer using image-to-image denoising diffusion implicit models
- 2023Discrete Diffusion Probabilistic Models for Symbolic Music Generation
- 2022Realistic Gramophone Noise Synthesis using a Diffusion Model
- 2024WaveTransfer: A Flexible End-to-end Multi-instrument Timbre Transfer with Diffusion
- 2024PeriodGrad: Towards Pitch-Controllable Neural Vocoder Based on a Diffusion Probabilistic Model
- 2022Full-band General Audio Synthesis with Score-based Diffusion
- 2023EDMSound: Spectrogram Based Diffusion Models for Efficient and High-Quality Audio Synthesis
- 2026DDSynth-RL: Audio Synthesizer Inversion via Discrete Diffusion with Reinforcement Learning