# BYOL for Audio: Exploring Pre-trained General-purpose Audio Representations

arXiv: https://arxiv.org/abs/2204.07402 · HTML: https://mlsanity.com/paper/2204.07402/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2022.

## Built on by (most cited first, 30 of 36)

- [Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play](https://mlsanity.com/paper/2303.12076/) (2023)
- [XKD: Cross-modal Knowledge Distillation with Domain Alignment for Video Representation Learning](https://mlsanity.com/paper/2211.13929/) (2022)
- [Self-supervised Audio Teacher-Student Transformer for Both Clip-level and Frame-level Tasks](https://mlsanity.com/paper/2306.04186/) (2023)
- [How to Scale Your EMA](https://mlsanity.com/paper/2307.13813/) (2023)
- [Masked Modeling Duo: Learning Representations by Encouraging Both Networks to Model the Input](https://mlsanity.com/paper/2210.14648/) (2022)
- [Benchmarking Representations for Speech, Music, and Acoustic Events](https://mlsanity.com/paper/2405.00934/) (2024)
- [animal2vec and MeerKAT: A self-supervised transformer for rare-event raw audio input and a large-scale reference dataset for bioacoustics](https://mlsanity.com/paper/2406.01253/) (2024)
- [Self-Supervised Learning for Few-Shot Bird Sound Classification](https://mlsanity.com/paper/2312.15824/) (2023)
- [Parameter-Efficient Continual Fine-Tuning: A Survey](https://mlsanity.com/paper/2504.13822/) (2025)
- [Foundation Models for Bioacoustics -- a Comparative Review](https://mlsanity.com/paper/2508.01277/) (2025)
- [Self-Supervised Learning for Speech Enhancement through Synthesis](https://mlsanity.com/paper/2211.02542/) (2022)
- [FunnyNet-W: Multimodal Learning of Funny Moments in Videos in the Wild](https://mlsanity.com/paper/2401.04210/) (2024)
- [Self-Supervised Learning for Anomalous Sound Detection](https://mlsanity.com/paper/2312.09578/) (2023)
- [Stem-JEPA: A Joint-Embedding Predictive Architecture for Musical Stem Compatibility Estimation](https://mlsanity.com/paper/2408.02514/) (2024)
- [Unmute the Patch Tokens: Rethinking Probing in Multi-Label Audio Classification](https://mlsanity.com/paper/2509.24901/) (2025)
- [Singer Identity Representation Learning using Self-Supervised Techniques](https://mlsanity.com/paper/2401.05064/) (2024)
- [MATPAC++: Enhanced Masked Latent Prediction for Self-Supervised Audio Representation Learning](https://mlsanity.com/paper/2508.12709/) (2025)
- [Audio Barlow Twins: Self-Supervised Audio Representation Learning](https://mlsanity.com/paper/2209.14345/) (2022)
- [On the Effect of Data-Augmentation on Local Embedding Properties in the Contrastive Learning of Music Audio Representations](https://mlsanity.com/paper/2401.08889/) (2024)
- [Enhanced Multimodal Content Moderation of Children's Videos using Audiovisual Fusion](https://mlsanity.com/paper/2405.06128/) (2024)
- [Investigating Design Choices in Joint-Embedding Predictive Architectures for General Audio Representation Learning](https://mlsanity.com/paper/2405.08679/) (2024)
- [Masked Latent Prediction and Classification for Self-Supervised Audio Representation Learning](https://mlsanity.com/paper/2502.12031/) (2025)
- [EmoSLLM: Parameter-Efficient Adaptation of LLMs for Speech Emotion Recognition](https://mlsanity.com/paper/2508.14130/) (2025)
- [Prototype based Masked Audio Model for Self-Supervised Learning of Sound Event Detection](https://mlsanity.com/paper/2409.17656/) (2024)
- [PESTO: Real-Time Pitch Estimation with Self-supervised Transposition-equivariant Objective](https://mlsanity.com/paper/2508.01488/) (2025)
- [Pretraining Representations for Bioacoustic Few-shot Detection using Supervised Contrastive Learning](https://mlsanity.com/paper/2309.00878/) (2023)
- [Self-Supervised Embeddings for Detecting Individual Symptoms of Depression](https://mlsanity.com/paper/2406.17229/) (2024)
- [The Computation of Generalized Embeddings for Underwater Acoustic Target Recognition using Contrastive Learning](https://mlsanity.com/paper/2505.12904/) (2025)
- [A Simple Method to Enhance Pre-trained Language Models with Speech Tokens for Classification](https://mlsanity.com/paper/2512.07571/) (2025)
- [On the choice of the optimal temporal support for audio classification with Pre-trained embeddings](https://mlsanity.com/paper/2312.14005/) (2023)

## Abstract (arXiv)

Pre-trained models are essential as feature extractors in modern machine learning systems in various domains. In this study, we hypothesize that representations effective for general audio tasks should provide multiple aspects of robust features of the input sound. For recognizing sounds regardless of perturbations such as varying pitch or timbre, features should be robust to these perturbations. For serving the diverse needs of tasks such as recognition of emotions or music genres, representations should provide multiple aspects of information, such as local and global features. To implement our principle, we propose a self-supervised learning method: Bootstrap Your Own Latent (BYOL) for Audio (BYOL-A, pronounced "viola"). BYOL-A pre-trains representations of the input sound invariant to audio data augmentations, which makes the learned representations robust to the perturbations of sounds. Whereas the BYOL-A encoder combines local and global features and calculates their statistics to make the representation provide multi-aspect information. As a result, the learned representations should provide robust and multi-aspect information to serve various needs of diverse tasks. We evaluated the general audio task performance of BYOL-A compared to previous state-of-the-art methods, and BYOL-A demonstrated generalizability with the best average result of 72.4% and the best VoxCeleb1 result of 57.6%. Extensive ablation experiments revealed that the BYOL-A encoder architecture contributes to most performance, and the final critical portion resorts to the BYOL framework and BYOL-A augmentations. Our code is available online at https://github.com/nttcslab/byol-a for future studies.