# Gradient-based Optimisation of Modulation Effects

arXiv: https://arxiv.org/abs/2601.04867 · HTML: https://mlsanity.com/paper/2601.04867/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Builds on (most cited first, 12 of 12)

- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [A Review of Differentiable Digital Signal Processing for Music & Speech Synthesis](https://mlsanity.com/paper/2308.15422/) (2023)
- [Sinusoidal Frequency Estimation by Gradient Descent](https://mlsanity.com/paper/2210.14476/) (2022)
- [Perceptual Loss Function for Neural Modelling of Audio Systems](https://mlsanity.com/paper/1911.08922/) (2019)
- [Differentiable Artificial Reverberation](https://mlsanity.com/paper/2105.13940/) (2021)
- [Differentiable All-pole Filters for Time-varying Audio Systems](https://mlsanity.com/paper/2404.07970/) (2024)
- [Modulation Extraction for LFO-driven Audio Effects](https://mlsanity.com/paper/2305.13262/) (2023)
- [Differentiable Grey-box Modelling of Phaser Effects using Frame-based Spectral Processing](https://mlsanity.com/paper/2306.01332/) (2023)
- [CONMOD: Controllable Neural Frame-based Modulation Effects](https://mlsanity.com/paper/2406.13935/) (2024)
- [FLAMO: An Open-Source Library for Frequency-Domain Differentiable Audio Processing](https://mlsanity.com/paper/2409.08723/) (2024)
- [Modulation Discovery with Differentiable Digital Signal Processing](https://mlsanity.com/paper/2510.06204/) (2025)

## Abstract (arXiv)

Modulation effects such as phasers, flangers and chorus effects are heavily used in conjunction with the electric guitar. Machine learning based emulation of analog modulation units has been investigated in recent years, but most methods have either been limited to one class of effect or suffer from a high computational cost or latency compared to canonical digital implementations. Here, we build on previous work and present a framework for modelling flanger, chorus and phaser effects based on differentiable digital signal processing. The model is trained in the time-frequency domain, but at inference operates in the time-domain, requiring zero latency. We investigate the challenges associated with gradient-based optimisation of such effects, and show that low-frequency weighting of loss functions avoids convergence to local minima when learning delay times. We show that when trained against analog effects units, sound output from the model is in some cases perceptually indistinguishable from the reference, but challenges still remain for effects with long delay times and feedback.