# Synthesizer Sound Matching Using Audio Spectrogram Transformers

arXiv: https://arxiv.org/abs/2407.16643 · HTML: https://mlsanity.com/paper/2407.16643/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 23 Jul 2024.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Efficient Training of Audio Transformers with Patchout](https://mlsanity.com/paper/2110.05069/) (2021)
- [Sequence-to-Sequence Piano Transcription with Transformers](https://mlsanity.com/paper/2107.09142/) (2021)
- [Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks](https://mlsanity.com/paper/1808.06719/) (2018)
- [DDX7: Differentiable FM Synthesis of Musical Instrument Sounds](https://mlsanity.com/paper/2208.06169/) (2022)
- [DiffMoog: a Differentiable Modular Synthesizer for Sound Matching](https://mlsanity.com/paper/2401.12570/) (2024)

## Built on by (most cited first, 4 of 4)

- [Audio synthesizer inversion in symmetric parameter spaces with approximately equivariant flow matching](https://mlsanity.com/paper/2506.07199/) (2025)
- [Modulation Discovery with Differentiable Digital Signal Processing](https://mlsanity.com/paper/2510.06204/) (2025)
- [Neural Proxies for Sound Synthesizers: Learning Perceptually Informed Preset Representations](https://mlsanity.com/paper/2509.07635/) (2025)
- [Off-manifold robustness in synthesizer inversion with joint distribution flow matching](https://mlsanity.com/paper/2609.29320/) (2026)

## Abstract (arXiv)

Systems for synthesizer sound matching, which automatically set the parameters of a synthesizer to emulate an input sound, have the potential to make the process of synthesizer programming faster and easier for novice and experienced musicians alike, whilst also affording new means of interaction with synthesizers. Considering the enormous variety of synthesizers in the marketplace, and the complexity of many of them, general-purpose sound matching systems that function with minimal knowledge or prior assumptions about the underlying synthesis architecture are particularly desirable. With this in mind, we introduce a synthesizer sound matching model based on the Audio Spectrogram Transformer. We demonstrate the viability of this model by training on a large synthetic dataset of randomly generated samples from the popular Massive synthesizer. We show that this model can reconstruct parameters of samples generated from a set of 16 parameters, highlighting its improved fidelity relative to multi-layer perceptron and convolutional neural network baselines. We also provide audio examples demonstrating the out-of-domain model performance in emulating vocal imitations, and sounds from other synthesizers and musical instruments.