# DiffMoog: a Differentiable Modular Synthesizer for Sound Matching

arXiv: https://arxiv.org/abs/2401.12570 · HTML: https://mlsanity.com/paper/2401.12570/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 23 Jan 2024.

## Builds on (most cited first, 18 of 18)

- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders](https://mlsanity.com/paper/1704.01279/) (2017)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [Noise2Music: Text-conditioned Music Generation with Diffusion Models](https://mlsanity.com/paper/2302.03917/) (2023)
- [GANSynth: Adversarial Neural Audio Synthesis](https://mlsanity.com/paper/1902.08710/) (2019)
- [CREPE: A Convolutional Representation for Pitch Estimation](https://mlsanity.com/paper/1802.06182/) (2018)
- [RAVE: A variational autoencoder for fast and high-quality neural audio synthesis](https://mlsanity.com/paper/2111.05011/) (2021)
- [Automatic multitrack mixing with a differentiable mixing console of neural audio effects](https://mlsanity.com/paper/2010.10291/) (2020)
- [SING: Symbol-to-Instrument Neural Generator](https://mlsanity.com/paper/1810.09785/) (2018)
- [Style Transfer of Audio Effects with Differentiable Signal Processing](https://mlsanity.com/paper/2207.08759/) (2022)
- [Universal audio synthesizer control with normalizing flows](https://mlsanity.com/paper/1907.00971/) (2019)
- [I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitch](https://mlsanity.com/paper/2012.04572/) (2020)
- [WGANSing: A Multi-Voice Singing Voice Synthesizer Based on the Wasserstein-GAN](https://mlsanity.com/paper/1903.10729/) (2019)
- [DDX7: Differentiable FM Synthesis of Musical Instrument Sounds](https://mlsanity.com/paper/2208.06169/) (2022)
- [InverSynth: Deep Estimation of Synthesizer Parameter Configurations from Audio Signals](https://mlsanity.com/paper/1812.06349/) (2018)
- [Differentiable Signal Processing With Black-Box Audio Effects](https://mlsanity.com/paper/2105.04752/) (2021)
- [Differentiable Wavetable Synthesis](https://mlsanity.com/paper/2111.10003/) (2021)

## Built on by (most cited first, 7 of 7)

- [Audio synthesizer inversion in symmetric parameter spaces with approximately equivariant flow matching](https://mlsanity.com/paper/2506.07199/) (2025)
- [Synthesizer Sound Matching Using Audio Spectrogram Transformers](https://mlsanity.com/paper/2407.16643/) (2024)
- [Modulation Discovery with Differentiable Digital Signal Processing](https://mlsanity.com/paper/2510.06204/) (2025)
- [WildFX: A DAW-Powered Pipeline for In-the-Wild Audio FX Graph Modeling](https://mlsanity.com/paper/2507.10534/) (2025)
- [Neural Proxies for Sound Synthesizers: Learning Perceptually Informed Preset Representations](https://mlsanity.com/paper/2509.07635/) (2025)
- [DDSynth-RL: Audio Synthesizer Inversion via Discrete Diffusion with Reinforcement Learning](https://mlsanity.com/paper/2608.03032/) (2026)
- [Evaluating Loss Functions in Differentiable Out-of-Domain Sound-Matching with Partial Parameter Distance](https://mlsanity.com/paper/2608.27698/) (2026)

## Abstract (arXiv)

This paper presents DiffMoog - a differentiable modular synthesizer with a comprehensive set of modules typically found in commercial instruments. Being differentiable, it allows integration into neural networks, enabling automated sound matching, to replicate a given audio input. Notably, DiffMoog facilitates modulation capabilities (FM/AM), low-frequency oscillators (LFOs), filters, envelope shapers, and the ability for users to create custom signal chains. We introduce an open-source platform that comprises DiffMoog and an end-to-end sound matching framework. This framework utilizes a novel signal-chain loss and an encoder network that self-programs its outputs to predict DiffMoogs parameters based on the user-defined modular architecture. Moreover, we provide insights and lessons learned towards sound matching using differentiable synthesis. Combining robust sound capabilities with a holistic platform, DiffMoog stands as a premier asset for expediting research in audio synthesis and machine learning.