# CLAR: Contrastive Learning of Auditory Representations

arXiv: https://arxiv.org/abs/2010.09542 · HTML: https://mlsanity.com/paper/2010.09542/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2020.

## Built on by (most cited first, 30 of 31)

- [BEATs: Audio Pre-Training with Acoustic Tokenizers](https://mlsanity.com/paper/2212.09058/) (2022)
- [Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2103.09410/) (2021)
- [Audio Self-supervised Learning: A Survey](https://mlsanity.com/paper/2203.01205/) (2022)
- [Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices](https://mlsanity.com/paper/2201.12382/) (2022)
- [Foundation Models for Music: A Survey](https://mlsanity.com/paper/2408.14340/) (2024)
- [Contrastive Audio-Language Learning for Music](https://mlsanity.com/paper/2208.12208/) (2022)
- [Perch 2.0: The Bittern Lesson for Bioacoustics](https://mlsanity.com/paper/2508.04665/) (2025)
- [COCOLA: Coherence-Oriented Contrastive Learning of Musical Audio Representations](https://mlsanity.com/paper/2404.16969/) (2024)
- [ASiT: Local-Global Audio Spectrogram vIsion Transformer for Event Classification](https://mlsanity.com/paper/2211.13189/) (2022)
- [A Contrastive Learning Approach to Mitigate Bias in Speech Models](https://mlsanity.com/paper/2406.14686/) (2024)
- [Amortised Invariance Learning for Contrastive Self-Supervision](https://mlsanity.com/paper/2302.12712/) (2023)
- [SpliceOut: A Simple and Efficient Audio Augmentation Method](https://mlsanity.com/paper/2110.00046/) (2021)
- [Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data](https://mlsanity.com/paper/2410.02056/) (2024)
- [Singer Identity Representation Learning using Self-Supervised Techniques](https://mlsanity.com/paper/2401.05064/) (2024)
- [Towards Proper Contrastive Self-supervised Learning Strategies For Music Audio Representation](https://mlsanity.com/paper/2207.04471/) (2022)
- [Audio Barlow Twins: Self-Supervised Audio Representation Learning](https://mlsanity.com/paper/2209.14345/) (2022)
- [Automatic Data Augmentation Selection and Parametrization in Contrastive Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2204.04170/) (2022)
- [MT-SLVR: Multi-Task Self-Supervised Learning for Transformation In(Variant) Representations](https://mlsanity.com/paper/2305.17191/) (2023)
- [Speech Analysis of Language Varieties in Italy](https://mlsanity.com/paper/2406.15862/) (2024)
- [Cross-Modal Contrastive Representation Learning for Audio-to-Image Generation](https://mlsanity.com/paper/2207.12121/) (2022)
- [Optimizing Audio Augmentations for Contrastive Learning of Health-Related Acoustic Signals](https://mlsanity.com/paper/2309.05843/) (2023)
- [Self-supervised Learning for Acoustic Few-Shot Classification](https://mlsanity.com/paper/2409.09647/) (2024)
- [AudioFormer: Audio Transformer learns audio feature representations from discrete acoustic codes](https://mlsanity.com/paper/2308.07221/) (2023)
- [Contrastive Learning from Synthetic Audio Doppelg"angers](https://mlsanity.com/paper/2406.05923/) (2024)
- [A Joint Learning Approach to Hardware Caching and Prefetching](https://mlsanity.com/paper/2510.10862/) (2025)
- [Resounding Acoustic Fields with Reciprocity](https://mlsanity.com/paper/2510.20602/) (2025)
- [The Impact of Spatiotemporal Augmentations on Self-Supervised Audiovisual Representation Learning](https://mlsanity.com/paper/2110.07082/) (2021)
- [Towards noise robust trigger-word detection with contrastive learning pre-task for fast on-boarding of new trigger-words](https://mlsanity.com/paper/2111.03971/) (2021)
- [Audio Contrastive-based Fine-tuning: Decoupling Representation Learning and Classification](https://mlsanity.com/paper/2309.11895/) (2023)
- [SS-DPPN: A self-supervised dual-path foundation model for the generalizable cardiac audio representation](https://mlsanity.com/paper/2510.10719/) (2025)

## Abstract (arXiv)

Learning rich visual representations using contrastive self-supervised learning has been extremely successful. However, it is still a major question whether we could use a similar approach to learn superior auditory representations. In this paper, we expand on prior work (SimCLR) to learn better auditory representations. We (1) introduce various data augmentations suitable for auditory data and evaluate their impact on predictive performance, (2) show that training with time-frequency audio features substantially improves the quality of the learned representations compared to raw signals, and (3) demonstrate that training with both supervised and contrastive losses simultaneously improves the learned representations compared to self-supervised pre-training followed by supervised fine-tuning. We illustrate that by combining all these methods and with substantially less labeled data, our framework (CLAR) achieves significant improvement on prediction performance compared to supervised approach. Moreover, compared to self-supervised approach, our framework converges faster with significantly better representations.