# Neural Proxies for Sound Synthesizers: Learning Perceptually Informed Preset Representations

arXiv: https://arxiv.org/abs/2509.07635 · HTML: https://mlsanity.com/paper/2509.07635/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 9 Sep 2025.

## Builds on (most cited first, 24 of 24)

- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [Efficient Training of Audio Transformers with Patchout](https://mlsanity.com/paper/2110.05069/) (2021)
- [Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders](https://mlsanity.com/paper/1704.01279/) (2017)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [DLOW: Domain Flow for Adaptation and Generalization](https://mlsanity.com/paper/1812.05418/) (2018)
- [Training Very Deep Networks](https://mlsanity.com/paper/1507.06228/) (2015)
- [Music2Latent: Consistency Autoencoders for Latent Audio Compression](https://mlsanity.com/paper/2408.06500/) (2024)
- [Style Transfer of Audio Effects with Differentiable Signal Processing](https://mlsanity.com/paper/2207.08759/) (2022)
- [DDX7: Differentiable FM Synthesis of Musical Instrument Sounds](https://mlsanity.com/paper/2208.06169/) (2022)
- [One Billion Audio Sounds from GPU-enabled Modular Synthesis](https://mlsanity.com/paper/2104.12922/) (2021)
- [Differentiable Signal Processing With Black-Box Audio Effects](https://mlsanity.com/paper/2105.04752/) (2021)
- [DawDreamer: Bridging the Gap Between Digital Audio Workstations and Python Interfaces](https://mlsanity.com/paper/2111.09931/) (2021)
- [Sound2Synth: Interpreting Sound via FM Synthesizer Parameters Estimation](https://mlsanity.com/paper/2205.03043/) (2022)
- [DiffMoog: a Differentiable Modular Synthesizer for Sound Matching](https://mlsanity.com/paper/2401.12570/) (2024)
- [Neural network gradient-based learning of black-box function interfaces](https://mlsanity.com/paper/1901.03995/) (2019)
- [Creative Text-to-Audio Generation via Synthesizer Programming](https://mlsanity.com/paper/2406.00294/) (2024)
- [Synthesizer Sound Matching Using Audio Spectrogram Transformers](https://mlsanity.com/paper/2407.16643/) (2024)
- [Synthesizer Preset Interpolation using Transformer Auto-Encoders](https://mlsanity.com/paper/2210.16984/) (2022)
- [SynthScribe: Deep Multimodal Tools for Synthesizer Sound Retrieval and Exploration](https://mlsanity.com/paper/2312.04690/) (2023)
- [Low-Complexity Audio Embedding Extractors](https://mlsanity.com/paper/2303.01879/) (2023)

## Abstract (arXiv)

Deep learning appears as an appealing solution for Automatic Synthesizer Programming (ASP), which aims to assist musicians and sound designers in programming sound synthesizers. However, integrating software synthesizers into training pipelines is challenging due to their potential non-differentiability. This work tackles this challenge by introducing a method to approximate arbitrary synthesizers. Specifically, we train a neural network to map synthesizer presets onto an audio embedding space derived from a pretrained model. This facilitates the definition of a neural proxy that produces compact yet effective representations, thereby enabling the integration of audio embedding loss into neural-based ASP systems for black-box synthesizers. We evaluate the representations derived by various pretrained audio models in the context of neural-based nASP and assess the effectiveness of several neural network architectures, including feedforward, recurrent, and transformer-based models, in defining neural proxies. We evaluate the proposed method using both synthetic and hand-crafted presets from three popular software synthesizers and assess its performance in a synthesizer sound matching downstream task. While the benefits of the learned representation are nuanced by resource requirements, encouraging results were obtained for all synthesizers, paving the way for future research into the application of synthesizer proxies for neural-based ASP systems.