# Modulation Discovery with Differentiable Digital Signal Processing

arXiv: https://arxiv.org/abs/2510.06204 · HTML: https://mlsanity.com/paper/2510.06204/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 7 Oct 2025.

## Builds on (most cited first, 20 of 20)

- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [A Neural Representation of Sketch Drawings](https://mlsanity.com/paper/1704.03477/) (2017)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [The Road Less Scheduled](https://mlsanity.com/paper/2405.15682/) (2024)
- [Natural Language Supervision for General-Purpose Audio Representations](https://mlsanity.com/paper/2309.05767/) (2023)
- [Neural Painters: A learned differentiable constraint for generating brushstroke paintings](https://mlsanity.com/paper/1904.08410/) (2019)
- [A Review of Differentiable Digital Signal Processing for Music & Speech Synthesis](https://mlsanity.com/paper/2308.15422/) (2023)
- [DDX7: Differentiable FM Synthesis of Musical Instrument Sounds](https://mlsanity.com/paper/2208.06169/) (2022)
- [Neural Waveshaping Synthesis](https://mlsanity.com/paper/2107.05050/) (2021)
- [One Billion Audio Sounds from GPU-enabled Modular Synthesis](https://mlsanity.com/paper/2104.12922/) (2021)
- [Sound2Synth: Interpreting Sound via FM Synthesizer Parameters Estimation](https://mlsanity.com/paper/2205.03043/) (2022)
- [DiffMoog: a Differentiable Modular Synthesizer for Sound Matching](https://mlsanity.com/paper/2401.12570/) (2024)
- [Differentiable All-pole Filters for Time-varying Audio Systems](https://mlsanity.com/paper/2404.07970/) (2024)
- [Modulation Extraction for LFO-driven Audio Effects](https://mlsanity.com/paper/2305.13262/) (2023)
- [Audio synthesizer inversion in symmetric parameter spaces with approximately equivariant flow matching](https://mlsanity.com/paper/2506.07199/) (2025)
- [Synthesizer Sound Matching Using Audio Spectrogram Transformers](https://mlsanity.com/paper/2407.16643/) (2024)
- [Differentiable Segmentation of Sequences](https://mlsanity.com/paper/2006.13105/) (2020)
- [SerumRNN: Step by Step Audio VST Effect Programming](https://mlsanity.com/paper/2104.03876/) (2021)
- [Steerable discovery of neural audio effects](https://mlsanity.com/paper/2112.02926/) (2021)

## Built on by (most cited first, 1 of 1)

- [Gradient-based Optimisation of Modulation Effects](https://mlsanity.com/paper/2601.04867/) (2026)

## Abstract (arXiv)

Modulations are a critical part of sound design and music production, enabling the creation of complex and evolving audio. Modern synthesizers provide envelopes, low frequency oscillators (LFOs), and more parameter automation tools that allow users to modulate the output with ease. However, determining the modulation signals used to create a sound is difficult, and existing sound-matching / parameter estimation systems are often uninterpretable black boxes or predict high-dimensional framewise parameter values without considering the shape, structure, and routing of the underlying modulation curves. We propose a neural sound-matching approach that leverages modulation extraction, constrained control signal parameterizations, and differentiable digital signal processing (DDSP) to discover the modulations present in a sound. We demonstrate the effectiveness of our approach on highly modulated synthetic and real audio samples, its applicability to different DDSP synth architectures, and investigate the trade-off it incurs between interpretability and sound-matching accuracy. We make our code and audio samples available and provide the trained DDSP synths in a VST plugin.