Audio synthesizer inversion in symmetric parameter spaces with approximately equivariant flow matching
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
Many audio synthesizers can produce the same signal given different parameter configurations, meaning the inversion from sound to parameters is an inherently ill-posed problem. We show that this is largely due to intrinsic symmetries of the synthesizer, and focus in particular on permutation invariance. First, we demonstrate on a synthetic task that regressing point estimates under permutation symmetry degrades performance, even when using a permutation-invariant loss function or symmetry-breaking heuristics. Then, viewing equivalent solutions as modes of a probability distribution, we show that a conditional generative model substantially improves performance. Further, acknowledging the invariance of the implicit parameter distribution, we find that performance is further improved by using a permutation equivariant continuous normalizing flow. To accommodate intricate symmetries in real synthesizers, we also propose a relaxed equivariance strategy that adaptively discovers relevant symmetries from data. Applying our method to Surge XT, a full-featured open source synthesizer used in real world audio production, we find our method outperforms regression and generative baselines across audio reconstruction metrics.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 37 of its 55 references that are in the map. First posted on arXiv 8 Jun 2025.
Builds on
37 of 55 references.
References · 37
- 2017Attention Is All You Needcited by 60,361
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2018Neural Ordinary Differential Equationscited by 3,779
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2016Density estimation using Real NVPcited by 2,042
- 2020On Layer Normalization in the Transformer Architecturecited by 922
- 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 879
- 2021AST: Audio Spectrogram Transformercited by 643
- 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
- 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 342
25 more
- 2023Multisample Flow Matching: Straightening Flows with Minibatch Couplingscited by 278
- 2016Using the Output Embedding to Improve Language Modelscited by 272
- 2020Equivariant Flows: Exact Likelihood Generative Learning for Symmetric Densitiescited by 250
- 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 225
- 2020DDSP: Differentiable Digital Signal Processingcited by 178
- 2018CREPE: A Convolutional Representation for Pitch Estimationcited by 110
- 2023Guided Flows for Generative Modeling and Decision Makingcited by 101
- 2021SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Datacited by 78
- 2019Deep Set Prediction Networkscited by 71
- 2023Equivariant Flow Matching with Hybrid Probability Transportcited by 60
- 2022Fast Point Cloud Generation with Straight Flowscited by 56
- 2024Mixed Continuous and Categorical Flow Matching for 3D De Novo Molecule Generationcited by 54
- 2019FSPool: Learning Set Representations with Featurewise Sort Poolingcited by 53
- 2024ET-Flow: Equivariant Flow-Matching for Molecular Conformer Generationcited by 43
- 2020Conditional Set Generation with Transformerscited by 32
- 2020Scalable Normalizing Flows for Permutation Invariant Densitiescited by 22
- 2023A Review of Differentiable Digital Signal Processing for Music & Speech Synthesiscited by 20
- 2020I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitchcited by 17
- 2020Set Prediction without Imposing Structure as Conditional Density Estimationcited by 12
- 2018InverSynth: Deep Estimation of Synthesizer Parameter Configurations from Audio Signalscited by 11
- 2023Unsupervised Harmonic Parameter Estimation Using Differentiable DSP and Spectral Optimal Transportcited by 8
- 2024DiffMoog: a Differentiable Modular Synthesizer for Sound Matchingcited by 7
- 2023Learning to Solve Inverse Problems for Perceptual Sound Matchingcited by 5
- 2023The Responsibility Problem in Neural Networks with Unordered Targetscited by 4
- 2024Synthesizer Sound Matching Using Audio Spectrogram Transformerscited by 4
Built on by
Citing papers per year: 2025: 1; 2026: 5.
6 papers.
- 2025Modulation Discovery with Differentiable Digital Signal Processingcited by 1
- 2026DDSynth-RL: Audio Synthesizer Inversion via Discrete Diffusion with Reinforcement Learningcited by 0
- 2026Equivariant Music Transformercited by 0
- 2026Diff2Mix: Controllable Music Mixing via Diffusion Models and Differentiable Audio Effectscited by 0
- 2026Off-manifold robustness in synthesizer inversion with joint distribution flow matchingcited by 0
- 2026Synth-JEPA: Joint Embedding Prediction for Renderer-Free Synthesizer Parameter Searchcited by 0
- 2026Synth-JEPA: Joint Embedding Prediction for Renderer-Free Synthesizer Parameter Search25 Sep 2026
- 2026Off-manifold robustness in synthesizer inversion with joint distribution flow matching24 Sep 2026
- 2026Diff2Mix: Controllable Music Mixing via Diffusion Models and Differentiable Audio Effects5 Aug 2026
- 2026Equivariant Music Transformer4 Aug 2026
- 2026DDSynth-RL: Audio Synthesizer Inversion via Discrete Diffusion with Reinforcement Learning4 Aug 2026
- 2025Modulation Discovery with Differentiable Digital Signal Processing7 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2022Solving Audio Inverse Problems with a Diffusion Model
- 2025FlowSynth: Instrument Generation Through Distributional Flow Matching and Test-Time Search
- 2025Abstract Sound Fusion with Unconditional Inversion Models
- 2026On the Geometry of Music Bandwidth Extension in Latent Spaces of Audio Codecs
- 2022Multi-instrument Music Synthesis with Spectrogram Diffusion
- 2023Controllable Music Production with Diffusion Models and Guidance Gradients
- 2026Audio-to-Audio via Diffusion Warm Initialization ↗
- 2023EDMSound: Spectrogram Based Diffusion Models for Efficient and High-Quality Audio Synthesis