# Feature-informed Embedding Space Regularization For Audio Classification

arXiv: https://arxiv.org/abs/2206.04850 · HTML: https://mlsanity.com/paper/2206.04850/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2022.

## Built on by (most cited first, 4 of 4)

- [Audio Embeddings as Teachers for Music Classification](https://mlsanity.com/paper/2306.17424/) (2023)
- [Embedding Compression for Teacher-to-Student Knowledge Transfer](https://mlsanity.com/paper/2402.06761/) (2024)
- [Chirp Localization via Fine-Tuned Transformer Model: A Proof-of-Concept Study](https://mlsanity.com/paper/2503.22713/) (2025)
- [S-SONDO: Self-Supervised Knowledge Distillation for General Audio Foundation Models](https://mlsanity.com/paper/2604.24933/) (2026)

## Abstract (arXiv)

Feature representations derived from models pre-trained on large-scale datasets have shown their generalizability on a variety of audio analysis tasks. Despite this generalizability, however, task-specific features can outperform if sufficient training data is available, as specific task-relevant properties can be learned. Furthermore, the complex pre-trained models bring considerable computational burdens during inference. We propose to leverage both detailed task-specific features from spectrogram input and generic pre-trained features by introducing two regularization methods that integrate the information of both feature classes. The workload is kept low during inference as the pre-trained features are only necessary for training. In experiments with the pre-trained features VGGish, OpenL3, and a combination of both, we show that the proposed methods not only outperform baseline methods, but also can improve state-of-the-art models on several audio classification tasks. The results also suggest that using the mixture of features performs better than using individual features.