# A general-purpose deep learning approach to model time-varying audio effects

arXiv: https://arxiv.org/abs/1905.06148 · HTML: https://mlsanity.com/paper/1905.06148/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2019.

## Built on by (most cited first, 8 of 8)

- [Modelling black-box audio effects with time-varying feature modulation](https://mlsanity.com/paper/2211.00497/) (2022)
- [Modulation Extraction for LFO-driven Audio Effects](https://mlsanity.com/paper/2305.13262/) (2023)
- [Distortion Audio Effects: Learning How to Recover the Clean Signal](https://mlsanity.com/paper/2202.01664/) (2022)
- [Adaptive Latent Space Tuning for Non-Stationary Distributions](https://mlsanity.com/paper/2105.03584/) (2021)
- [CONMOD: Controllable Neural Frame-based Modulation Effects](https://mlsanity.com/paper/2406.13935/) (2024)
- [Parametric Neural Amp Modeling with Active Learning](https://mlsanity.com/paper/2507.02109/) (2025)
- [Parametric Neural Amp Modeling with Active Learning](https://mlsanity.com/paper/2509.26564/) (2025)
- [Time-Varying Audio Effect Modeling by End-to-End Adversarial Training](https://mlsanity.com/paper/2512.15313/) (2025)

## Abstract (arXiv)

Audio processors whose parameters are modified periodically over time are often referred as time-varying or modulation based audio effects. Most existing methods for modeling these type of effect units are often optimized to a very specific circuit and cannot be efficiently generalized to other time-varying effects. Based on convolutional and recurrent neural networks, we propose a deep learning architecture for generic black-box modeling of audio processors with long-term memory. We explore the capabilities of deep neural networks to learn such long temporal dependencies and we show the network modeling various linear and nonlinear, time-varying and time-invariant audio effects. In order to measure the performance of the model, we propose an objective metric based on the psychoacoustics of modulation frequency perception. We also analyze what the model is actually learning and how the given task is accomplished.