# I'm Sorry for Your Loss: Spectrally-Based Audio Distances Are Bad at Pitch

arXiv: https://arxiv.org/abs/2012.04572 · HTML: https://mlsanity.com/paper/2012.04572/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2020.

## Built on by (most cited first, 17 of 17)

- [DDX7: Differentiable FM Synthesis of Musical Instrument Sounds](https://mlsanity.com/paper/2208.06169/) (2022)
- [One Billion Audio Sounds from GPU-enabled Modular Synthesis](https://mlsanity.com/paper/2104.12922/) (2021)
- [Unsupervised Harmonic Parameter Estimation Using Differentiable DSP and Spectral Optimal Transport](https://mlsanity.com/paper/2312.14507/) (2023)
- [DiffMoog: a Differentiable Modular Synthesizer for Sound Matching](https://mlsanity.com/paper/2401.12570/) (2024)
- [Audio synthesizer inversion in symmetric parameter spaces with approximately equivariant flow matching](https://mlsanity.com/paper/2506.07199/) (2025)
- [Towards Lightweight Controllable Audio Synthesis with Conditional Implicit Neural Representations](https://mlsanity.com/paper/2111.08462/) (2021)
- [Improving Speech Enhancement through Fine-Grained Speech Characteristics](https://mlsanity.com/paper/2207.00237/) (2022)
- [Differentiable Modal Synthesis for Physical Modeling of Planar String Sound and Motion Simulation](https://mlsanity.com/paper/2407.05516/) (2024)
- [Real-time Timbre Remapping with Differentiable DSP](https://mlsanity.com/paper/2407.04547/) (2024)
- [Fast Differentiable Modal Simulation of Non-linear Strings, Membranes, and Plates](https://mlsanity.com/paper/2505.05940/) (2025)
- [Mesostructures: Beyond Spectrogram Loss in Differentiable Time-Frequency Analysis](https://mlsanity.com/paper/2301.10183/) (2023)
- [TAPLoss: A Temporal Acoustic Parameter Loss for Speech Enhancement](https://mlsanity.com/paper/2302.08088/) (2023)
- [PAAPLoss: A Phonetic-Aligned Acoustic Parameter Loss for Speech Enhancement](https://mlsanity.com/paper/2302.08095/) (2023)
- [R-MelNet: Reduced Mel-Spectral Modeling for Neural TTS](https://mlsanity.com/paper/2206.15276/) (2022)
- [Architecture and Affordances of PLAUD: Performative Latents and Unsupervised DDSP](https://mlsanity.com/paper/2608.13724/) (2026)
- [Evaluating Loss Functions in Differentiable Out-of-Domain Sound-Matching with Partial Parameter Distance](https://mlsanity.com/paper/2608.27698/) (2026)
- [Off-manifold robustness in synthesizer inversion with joint distribution flow matching](https://mlsanity.com/paper/2609.29320/) (2026)

## Abstract (arXiv)

Growing research demonstrates that synthetic failure modes imply poor generalization. We compare commonly used audio-to-audio losses on a synthetic benchmark, measuring the pitch distance between two stationary sinusoids. The results are surprising: many have poor sense of pitch direction. These shortcomings are exposed using simple rank assumptions. Our task is trivial for humans but difficult for these audio distances, suggesting significant progress can be made in self-supervised audio learning by improving current losses.