# Perceptual-Neural-Physical Sound Matching

arXiv: https://arxiv.org/abs/2301.02886 · HTML: https://mlsanity.com/paper/2301.02886/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 22 references that are in the map.
- First posted on arXiv 7 Jan 2023.

## Builds on (most cited first, 8 of 8)

- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [LEAF: A Learnable Frontend for Audio Classification](https://mlsanity.com/paper/2101.08596/) (2021)
- [Kymatio: Scattering Transforms in Python](https://mlsanity.com/paper/1812.11214/) (2018)
- [Universal audio synthesizer control with normalizing flows](https://mlsanity.com/paper/1907.00971/) (2019)
- [Joint Time-Frequency Scattering](https://mlsanity.com/paper/1807.08869/) (2018)
- [Time-Frequency Scattering Accurately Models Auditory Similarities Between Instrumental Playing Techniques](https://mlsanity.com/paper/2007.10926/) (2020)
- [wav2shape: Hearing the Shape of a Drum Machine](https://mlsanity.com/paper/2007.10299/) (2020)

## Built on by (most cited first, 1 of 1)

- [Evaluating Loss Functions in Differentiable Out-of-Domain Sound-Matching with Partial Parameter Distance](https://mlsanity.com/paper/2608.27698/) (2026)

## Abstract (arXiv)

Sound matching algorithms seek to approximate a target waveform by parametric audio synthesis. Deep neural networks have achieved promising results in matching sustained harmonic tones. However, the task is more challenging when targets are nonstationary and inharmonic, e.g., percussion. We attribute this problem to the inadequacy of loss function. On one hand, mean square error in the parametric domain, known as "P-loss", is simple and fast but fails to accommodate the differing perceptual significance of each parameter. On the other hand, mean square error in the spectrotemporal domain, known as "spectral loss", is perceptually motivated and serves in differentiable digital signal processing (DDSP). Yet, spectral loss is a poor predictor of pitch intervals and its gradient may be computationally expensive; hence a slow convergence. Against this conundrum, we present Perceptual-Neural-Physical loss (PNP). PNP is the optimal quadratic approximation of spectral loss while being as fast as P-loss during training. We instantiate PNP with physical modeling synthesis as decoder and joint time-frequency scattering transform (JTFS) as spectral representation. We demonstrate its potential on matching synthetic drum sounds in comparison with other loss functions.