# Pretraining Representations for Bioacoustic Few-shot Detection using Supervised Contrastive Learning

arXiv: https://arxiv.org/abs/2309.00878 · HTML: https://mlsanity.com/paper/2309.00878/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 2 Sep 2023.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?](https://mlsanity.com/paper/2003.11539/) (2020)
- [Automatic acoustic detection of birds through deep learning: the first Bird Audio Detection challenge](https://mlsanity.com/paper/1807.05812/) (2018)
- [Multi-Format Contrastive Learning of Audio Representations](https://mlsanity.com/paper/2103.06508/) (2021)
- [BYOL for Audio: Exploring Pre-trained General-purpose Audio Representations](https://mlsanity.com/paper/2204.07402/) (2022)
- [Unsupervised Contrastive Learning of Sound Event Representations](https://mlsanity.com/paper/2011.07616/) (2020)
- [MetaAudio: A Few-Shot Audio Classification Benchmark](https://mlsanity.com/paper/2204.02121/) (2022)
- [Learning to detect an animal sound from five examples](https://mlsanity.com/paper/2305.13210/) (2023)
- [Few-shot bioacoustic event detection at the DCASE 2022 challenge](https://mlsanity.com/paper/2207.07911/) (2022)
- [Who calls the shots? Rethinking Few-Shot Learning for Audio](https://mlsanity.com/paper/2110.09600/) (2021)
- [Proposal-based Few-shot Sound Event Detection for Speech and Environmental Sounds with Perceivers](https://mlsanity.com/paper/2107.13616/) (2021)
- [SoundCLR: Contrastive Learning of Representations For Improved Environmental Sound Classification](https://mlsanity.com/paper/2103.01929/) (2021)

## Built on by (most cited first, 2 of 2)

- [Regularized Contrastive Pre-training for Few-shot Bioacoustic Sound Detection](https://mlsanity.com/paper/2309.08971/) (2023)
- [SpurAudio: A Benchmark for Studying Shortcut Learning in Few-Shot Audio Classification](https://mlsanity.com/paper/2605.13672/) (2026)

## Abstract (arXiv)

Deep learning has been widely used recently for sound event detection and classification. Its success is linked to the availability of sufficiently large datasets, possibly with corresponding annotations when supervised learning is considered. In bioacoustic applications, most tasks come with few labelled training data, because annotating long recordings is time consuming and costly. Therefore supervised learning is not the best suited approach to solve bioacoustic tasks. The bioacoustic community recasted the problem of sound event detection within the framework of few-shot learning, i.e. training a system with only few labeled examples. The few-shot bioacoustic sound event detection task in the DCASE challenge focuses on detecting events in long audio recordings given only five annotated examples for each class of interest. In this paper, we show that learning a rich feature extractor from scratch can be achieved by leveraging data augmentation using a supervised contrastive learning framework. We highlight the ability of this framework to transfer well for five-shot event detection on previously unseen classes in the training data. We obtain an F-score of 63.46% on the validation set and 42.7% on the test set, ranking second in the DCASE challenge. We provide an ablation study for the critical choices of data augmentation techniques as well as for the learning strategy applied on the training set.