# SoundCLR: Contrastive Learning of Representations For Improved Environmental Sound Classification

arXiv: https://arxiv.org/abs/2103.01929 · HTML: https://mlsanity.com/paper/2103.01929/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 47 references that are in the map.
- First posted on arXiv 2 Mar 2021.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [Understanding the Behaviour of Contrastive Loss](https://mlsanity.com/paper/2012.09740/) (2020)
- [Large Margin Deep Networks for Classification](https://mlsanity.com/paper/1803.05598/) (2018)
- [Analyzing and Improving Representations with the Soft Nearest Neighbor Loss](https://mlsanity.com/paper/1902.01889/) (2019)
- [Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification](https://mlsanity.com/paper/1608.04363/) (2016)
- [Learning from Between-class Examples for Deep Sound Recognition](https://mlsanity.com/paper/1711.10282/) (2017)
- [Rethinking CNN Models for Audio Classification](https://mlsanity.com/paper/2007.11154/) (2020)
- [ESResNet: Environmental Sound Classification Based on Visual Domain Models](https://mlsanity.com/paper/2004.07301/) (2020)
- [Deep Convolutional Neural Network with Mixup for Environmental Sound Classification](https://mlsanity.com/paper/1808.08405/) (2018)
- [Convolutional Recurrent Neural Networks for Bird Audio Detection](https://mlsanity.com/paper/1703.02317/) (2017)
- [Learning Environmental Sounds with Multi-scale Convolutional Neural Network](https://mlsanity.com/paper/1803.10219/) (2018)

## Built on by (most cited first, 4 of 4)

- [Real-Time Idling Vehicles Detection using Combined Audio-Visual Deep Learning](https://mlsanity.com/paper/2305.14579/) (2023)
- [Pretraining Respiratory Sound Representations using Metadata and Contrastive Learning](https://mlsanity.com/paper/2210.16192/) (2022)
- [Pretraining Representations for Bioacoustic Few-shot Detection using Supervised Contrastive Learning](https://mlsanity.com/paper/2309.00878/) (2023)
- [Towards noise robust trigger-word detection with contrastive learning pre-task for fast on-boarding of new trigger-words](https://mlsanity.com/paper/2111.03971/) (2021)

## Abstract (arXiv)

Environmental Sound Classification (ESC) is a challenging field of research in non-speech audio processing. Most of current research in ESC focuses on designing deep models with special architectures tailored for specific audio datasets, which usually cannot exploit the intrinsic patterns in the data. However recent studies have surprisingly shown that transfer learning from models trained on ImageNet is a very effective technique in ESC. Herein, we propose SoundCLR, a supervised contrastive learning method for effective environment sound classification with state-of-the-art performance, which works by learning representations that disentangle the samples of each class from those of other classes. Our deep network models are trained by combining a contrastive loss that contributes to a better probability output by the classification layer with a cross-entropy loss on the output of the classifier layer to map the samples to their respective 1-hot encoded labels. Due to the comparatively small sizes of the available environmental sound datasets, we propose and exploit a transfer learning and strong data augmentation pipeline and apply the augmentations on both the sound signals and their log-mel spectrograms before inputting them to the model. Our experiments show that our masking based augmentation technique on the log-mel spectrograms can significantly improve the recognition performance. Our extensive benchmark experiments show that our hybrid deep network models trained with combined contrastive and cross-entropy loss achieved the state-of-the-art performance on three benchmark datasets ESC-10, ESC-50, and US8K with validation accuracies of 99.75%, 93.4%, and 86.49% respectively. The ensemble version of our models also outperforms other top ensemble methods. The code is available at https://github.com/alireza-nasiri/SoundCLR.