# Investigating the Sensitivity of Pre-trained Audio Embeddings to Common Effects

arXiv: https://arxiv.org/abs/2501.15900 · HTML: https://mlsanity.com/paper/2501.15900/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 14 references that are in the map.
- First posted on arXiv 27 Jan 2025.

## Builds on (most cited first, 7 of 7)

- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2103.09410/) (2021)
- [Natural Language Supervision for General-Purpose Audio Representations](https://mlsanity.com/paper/2309.05767/) (2023)
- [Gender Bias in Depression Detection Using Audio Features](https://mlsanity.com/paper/2010.15120/) (2020)
- [Data Augmentation for Instrument Classification Robust to Audio Effects](https://mlsanity.com/paper/1907.08520/) (2019)
- [A Study on Robustness to Perturbations for Representations of Environmental Sound](https://mlsanity.com/paper/2203.10425/) (2022)
- [Transfer Learning and Bias Correction with Pre-trained Audio Embeddings](https://mlsanity.com/paper/2307.10834/) (2023)

## Built on by (most cited first, 3 of 3)

- [From Diet to Free Lunch: Estimating Auxiliary Signal Properties using Dynamic Pruning Masks in Speech Enhancement Networks](https://mlsanity.com/paper/2602.10666/) (2026)
- [Probing Low-Level Acoustic Attribute Encoding in CLAP Audio Embeddings](https://mlsanity.com/paper/2607.03806/) (2026)
- [Exploring How Audio Effects Alter Emotion with Foundation Models](https://mlsanity.com/paper/2509.15151/) (2025)

## Abstract (arXiv)

In recent years, foundation models have significantly advanced data-driven systems across various domains. Yet, their underlying properties, especially when functioning as feature extractors, remain under-explored. In this paper, we investigate the sensitivity to audio effects of audio embeddings extracted from widely-used foundation models, including OpenL3, PANNs, and CLAP. We focus on audio effects as the source of sensitivity due to their prevalent presence in large audio datasets. By applying parameterized audio effects (gain, low-pass filtering, reverberation, and bitcrushing), we analyze the correlation between the deformation trajectories and the effect strength in the embedding space. We propose to quantify the dimensionality and linearizability of the deformation trajectories induced by audio effects using canonical correlation analysis. We find that there exists a direction along which the embeddings move monotonically as the audio effect strength increases, but that the subspace containing the displacements is generally high-dimensional. This shows that pre-trained audio embeddings do not globally linearize the effects. Our empirical results on instrument classification downstream tasks confirm that projecting out the estimated deformation directions cannot generally improve the robustness of pre-trained embeddings to audio effects.