# Differentiable Signal Processing With Black-Box Audio Effects

arXiv: https://arxiv.org/abs/2105.04752 · HTML: https://mlsanity.com/paper/2105.04752/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 49 references that are in the map.
- First posted on arXiv 11 May 2021.

## Builds on (most cited first, 9 of 9)

- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [Deep Learning for Tube Amplifier Emulation](https://mlsanity.com/paper/1811.00334/) (2018)
- [Metric Learning vs Classification for Disentangled Music Representation Learning](https://mlsanity.com/paper/2008.03729/) (2020)
- [A Feature Learning Siamese Model for Intelligent Control of the Dynamic Range Compressor](https://mlsanity.com/paper/1905.01022/) (2019)
- [Modeling of nonlinear audio effects with end-to-end deep neural networks](https://mlsanity.com/paper/1810.06603/) (2018)
- [Neural network gradient-based learning of black-box function interfaces](https://mlsanity.com/paper/1901.03995/) (2019)
- [SignalTrain: Profiling Audio Compressors with Deep Neural Networks](https://mlsanity.com/paper/1905.11928/) (2019)

## Built on by (most cited first, 10 of 10)

- [Music Mixing Style Transfer: A Contrastive Learning Approach to Disentangle Audio Effects](https://mlsanity.com/paper/2211.02247/) (2022)
- [DiffMoog: a Differentiable Modular Synthesizer for Sound Matching](https://mlsanity.com/paper/2401.12570/) (2024)
- [End-to-end Music Remastering System Using Self-supervised and Adversarial Training](https://mlsanity.com/paper/2202.08520/) (2022)
- [Differentiable Wavetable Synthesis](https://mlsanity.com/paper/2111.10003/) (2021)
- [Automatic DJ Transitions with Differentiable Audio Effects and Generative Adversarial Networks](https://mlsanity.com/paper/2110.06525/) (2021)
- [Attention-Based Efficient Breath Sound Removal in Studio Audio Recordings](https://mlsanity.com/paper/2409.04949/) (2024)
- [MetaSpeech: Speech Effects Switch Along with Environment for Metaverse](https://mlsanity.com/paper/2210.13811/) (2022)
- [Style Transfer for Non-differentiable Audio Effects](https://mlsanity.com/paper/2309.17125/) (2023)
- [Neural Proxies for Sound Synthesizers: Learning Perceptually Informed Preset Representations](https://mlsanity.com/paper/2509.07635/) (2025)
- [CATRF: Codec-Adaptive TriPlane Radiance Fields for Volumetric Content Delivery](https://mlsanity.com/paper/2605.18054/) (2026)

## Abstract (arXiv)

We present a data-driven approach to automate audio signal processing by incorporating stateful third-party, audio effects as layers within a deep neural network. We then train a deep encoder to analyze input audio and control effect parameters to perform the desired signal manipulation, requiring only input-target paired audio data as supervision. To train our network with non-differentiable black-box effects layers, we use a fast, parallel stochastic gradient approximation scheme within a standard auto differentiation graph, yielding efficient end-to-end backpropagation. We demonstrate the power of our approach with three separate automatic audio production applications: tube amplifier emulation, automatic removal of breaths and pops from voice recordings, and automatic music mastering. We validate our results with a subjective listening test, showing our approach not only can enable new automatic audio effects tasks, but can yield results comparable to a specialized, state-of-the-art commercial solution for music mastering.