# Representation Learning for the Automatic Indexing of Sound Effects Libraries

arXiv: https://arxiv.org/abs/2208.09096 · HTML: https://mlsanity.com/paper/2208.09096/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 81 references that are in the map.
- First posted on arXiv 18 Aug 2022.

## Builds on (most cited first, 30 of 33)

- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Circle Loss: A Unified Perspective of Pair Similarity Optimization](https://mlsanity.com/paper/2002.10857/) (2020)
- [Dynamic Few-Shot Visual Learning without Forgetting](https://mlsanity.com/paper/1804.09458/) (2018)
- [Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning](https://mlsanity.com/paper/1904.06627/) (2019)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [Sampling Matters in Deep Embedding Learning](https://mlsanity.com/paper/1706.07567/) (2017)
- [Contrastive Learning of General-Purpose Audio Representations](https://mlsanity.com/paper/2010.10915/) (2020)
- [Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2103.09410/) (2021)
- [Improved Embeddings with Easy Positive Triplet Mining](https://mlsanity.com/paper/1904.04370/) (2019)
- [Towards Learning Universal Audio Representations](https://mlsanity.com/paper/2111.12124/) (2021)
- [MusPy: A Toolkit for Symbolic Music Generation](https://mlsanity.com/paper/2008.01951/) (2020)
- [BYOL for Audio: Exploring Pre-trained General-purpose Audio Representations](https://mlsanity.com/paper/2204.07402/) (2022)
- [Unsupervised Contrastive Learning of Sound Event Representations](https://mlsanity.com/paper/2011.07616/) (2020)
- [General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline](https://mlsanity.com/paper/1807.09902/) (2018)
- [Cross-Dataset Collaborative Learning for Semantic Segmentation in Autonomous Driving](https://mlsanity.com/paper/2103.11351/) (2021)
- [Leveraging Hierarchical Structures for Few-Shot Musical Instrument Recognition](https://mlsanity.com/paper/2107.07029/) (2021)
- [Who calls the shots? Rethinking Few-Shot Learning for Audio](https://mlsanity.com/paper/2110.09600/) (2021)
- [Transformer based unsupervised pre-training for acoustic representation learning](https://mlsanity.com/paper/2007.14602/) (2020)
- [Metric Learning vs Classification for Disentangled Music Representation Learning](https://mlsanity.com/paper/2008.03729/) (2020)
- [Cross-task learning for audio tagging, sound event detection and spatial localization: DCASE 2019 baseline systems](https://mlsanity.com/paper/1904.03476/) (2019)
- [Semi-Supervised Audio Classification with Partially Labeled Data](https://mlsanity.com/paper/2111.12761/) (2021)
- [Improving Music Performance Assessment with Contrastive Learning](https://mlsanity.com/paper/2108.01711/) (2021)

## Abstract (arXiv)

Labeling and maintaining a commercial sound effects library is a time-consuming task exacerbated by databases that continually grow in size and undergo taxonomy updates. Moreover, sound search and taxonomy creation are complicated by non-uniform metadata, an unrelenting problem even with the introduction of a new industry standard, the Universal Category System. To address these problems and overcome dataset-dependent limitations that inhibit the successful training of deep learning models, we pursue representation learning to train generalized embeddings that can be used for a wide variety of sound effects libraries and are a taxonomy-agnostic representation of sound. We show that a task-specific but dataset-independent representation can successfully address data issues such as class imbalance, inconsistent class labels, and insufficient dataset size, outperforming established representations such as OpenL3. Detailed experimental results show the impact of metric learning approaches and different cross-dataset training methods on representational effectiveness.