# Style Transfer of Audio Effects with Differentiable Signal Processing

arXiv: https://arxiv.org/abs/2207.08759 · HTML: https://mlsanity.com/paper/2207.08759/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2022.

## Built on by (most cited first, 18 of 18)

- [Music Mixing Style Transfer: A Contrastive Learning Approach to Disentangle Audio Effects](https://mlsanity.com/paper/2211.02247/) (2022)
- [Self-Supervised Audio-Visual Soundscape Stylization](https://mlsanity.com/paper/2409.14340/) (2024)
- [DiffMoog: a Differentiable Modular Synthesizer for Sound Matching](https://mlsanity.com/paper/2401.12570/) (2024)
- [Differentiable All-pole Filters for Time-varying Audio Systems](https://mlsanity.com/paper/2404.07970/) (2024)
- [Learning to Highlight Audio by Watching Movies](https://mlsanity.com/paper/2505.12154/) (2025)
- [Unsupervised Estimation of Nonlinear Audio Effects: Comparing Diffusion-Based and Adversarial approaches](https://mlsanity.com/paper/2504.04751/) (2025)
- [Open-Amp: Synthetic Data Framework for Audio Effect Foundation Models](https://mlsanity.com/paper/2411.14972/) (2024)
- [The Role of Communication and Reference Songs in the Mixing Process: Insights from Professional Mix Engineers](https://mlsanity.com/paper/2309.03404/) (2023)
- [Automatic Equalization for Individual Instrument Tracks Using Convolutional Neural Networks](https://mlsanity.com/paper/2407.16691/) (2024)
- [Conditional Flow Matching for Visually-Guided Acoustic Highlighting](https://mlsanity.com/paper/2602.03762/) (2026)
- [Style Transfer for Non-differentiable Audio Effects](https://mlsanity.com/paper/2309.17125/) (2023)
- [Improving Inference-Time Optimisation for Vocal Effects Style Transfer with a Gaussian Prior](https://mlsanity.com/paper/2505.11315/) (2025)
- [WildFX: A DAW-Powered Pipeline for In-the-Wild Audio FX Graph Modeling](https://mlsanity.com/paper/2507.10534/) (2025)
- [Neural Proxies for Sound Synthesizers: Learning Perceptually Informed Preset Representations](https://mlsanity.com/paper/2509.07635/) (2025)
- [An Interpretable, Controllable Time-Varying IIR Denoiser for On-Device Assistive Hearing](https://mlsanity.com/paper/2603.02794/) (2026)
- [Decomposer: Learning to Decompile Symbolic Music to Programs](https://mlsanity.com/paper/2607.01849/) (2026)
- [Clean2FX: Label-conditioned modeling for clean-to-effect guitar audio transformations](https://mlsanity.com/paper/2607.08863/) (2026)
- [Diff2Mix: Controllable Music Mixing via Diffusion Models and Differentiable Audio Effects](https://mlsanity.com/paper/2608.05442/) (2026)

## Abstract (arXiv)

We present a framework that can impose the audio effects and production style from one recording to another by example with the goal of simplifying the audio production process. We train a deep neural network to analyze an input recording and a style reference recording, and predict the control parameters of audio effects used to render the output. In contrast to past work, we integrate audio effects as differentiable operators in our framework, perform backpropagation through audio effects, and optimize end-to-end using an audio-domain loss. We use a self-supervised training strategy enabling automatic control of audio effects without the use of any labeled or paired training data. We survey a range of existing and new approaches for differentiable signal processing, showing how each can be integrated into our framework while discussing their trade-offs. We evaluate our approach on both speech and music tasks, demonstrating that our approach generalizes both to unseen recordings and even to sample rates different than those seen during training. Our approach produces convincing production style transfer results with the ability to transform input recordings to produced recordings, yielding audio effect control parameters that enable interpretability and user interaction.