# A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences

arXiv: https://arxiv.org/abs/2001.04460 · HTML: https://mlsanity.com/paper/2001.04460/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jan 2020.

## Built on by (most cited first, 23 of 23)

- [DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data](https://mlsanity.com/paper/2306.09344/) (2023)
- [Taming Visually Guided Sound Generation](https://mlsanity.com/paper/2110.08791/) (2021)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [CDPAM: Contrastive learning for perceptual audio similarity](https://mlsanity.com/paper/2102.05109/) (2021)
- [PEAVS: Perceptual Evaluation of Audio-Visual Synchrony Grounded in Viewers' Opinion Scores](https://mlsanity.com/paper/2404.07336/) (2024)
- [One Billion Audio Sounds from GPU-enabled Modular Synthesis](https://mlsanity.com/paper/2104.12922/) (2021)
- [Enrollment-stage Backdoor Attacks on Speaker Recognition Systems via Adversarial Ultrasound](https://mlsanity.com/paper/2306.16022/) (2023)
- [Challenges in creative generative models for music: a divergence maximization perspective](https://mlsanity.com/paper/2211.08856/) (2022)
- [Continuous descriptor-based control for deep audio synthesis](https://mlsanity.com/paper/2302.13542/) (2023)
- [Perceptual-based deep-learning denoiser as a defense against adversarial attacks on ASR systems](https://mlsanity.com/paper/2107.05222/) (2021)
- [Automatic Evaluation of Speaker Similarity](https://mlsanity.com/paper/2207.00344/) (2022)
- [TAPLoss: A Temporal Acoustic Parameter Loss for Speech Enhancement](https://mlsanity.com/paper/2302.08088/) (2023)
- [PAAPLoss: A Phonetic-Aligned Acoustic Parameter Loss for Speech Enhancement](https://mlsanity.com/paper/2302.08095/) (2023)
- [Localization of Just Noticeable Difference for Image Compression](https://mlsanity.com/paper/2306.07678/) (2023)
- [Predicting pairwise preferences between TTS audio stimuli using parallel ratings data and anti-symmetric twin neural networks](https://mlsanity.com/paper/2209.11003/) (2022)
- [CORN: Co-Trained Full- And No-Reference Speech Quality Assessment](https://mlsanity.com/paper/2310.09388/) (2023)
- [The Interpretation Gap in Text-to-Music Generation Models](https://mlsanity.com/paper/2407.10328/) (2024)
- [JSQA: Speech Quality Assessment with Perceptually-Inspired Contrastive Pretraining Based on JND Audio Pairs](https://mlsanity.com/paper/2507.11636/) (2025)
- [ID-Sim: An Identity-Focused Similarity Metric](https://mlsanity.com/paper/2604.05039/) (2026)
- [SEAM: Shortcut-Aware Real-Time Detection of Scripted vs. Spontaneous Speech for Interview Guardrails](https://mlsanity.com/paper/2606.06837/) (2026)
- [SEOFP-NET: Compression and Acceleration of Deep Neural Networks for Speech Enhancement Using Sign-Exponent-Only Floating-Points](https://mlsanity.com/paper/2111.04436/) (2021)
- [Abstract Sound Fusion with Unconditional Inversion Models](https://mlsanity.com/paper/2506.11811/) (2025)
- [Quieter Than the Room: Representation Drift and Task Robustness in Speech Encoders](https://mlsanity.com/paper/2609.27195/) (2026)

## Abstract (arXiv)

Many audio processing tasks require perceptual assessment. The ``gold standard`` of obtaining human judgments is time-consuming, expensive, and cannot be used as an optimization criterion. On the other hand, automated metrics are efficient to compute but often correlate poorly with human judgment, particularly for audio differences at the threshold of human detection. In this work, we construct a metric by fitting a deep neural network to a new large dataset of crowdsourced human judgments. Subjects are prompted to answer a straightforward, objective question: are two recordings identical or not? These pairs are algorithmically generated under a variety of perturbations, including noise, reverb, and compression artifacts; the perturbation space is probed with the goal of efficiently identifying the just-noticeable difference (JND) level of the subject. We show that the resulting learned metric is well-calibrated with human judgments, outperforming baseline methods. Since it is a deep network, the metric is differentiable, making it suitable as a loss function for other tasks. Thus, simply replacing an existing loss (e.g., deep feature loss) with our metric yields significant improvement in a denoising network, as measured by subjective pairwise comparison.