# MAP-Music2Vec: A Simple and Effective Baseline for Self-Supervised Music Audio Representation Learning

arXiv: https://arxiv.org/abs/2212.02508 · HTML: https://mlsanity.com/paper/2212.02508/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 15 references that are in the map.
- First posted on arXiv 5 Dec 2022.

## Builds on (most cited first, 10 of 10)

- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2103.09410/) (2021)
- [Codified audio language modeling learns useful representations for music information retrieval](https://mlsanity.com/paper/2107.05677/) (2021)
- [musicnn: Pre-trained convolutional neural networks for music audio tagging](https://mlsanity.com/paper/1909.06654/) (2019)
- [Transfer learning for music classification and regression tasks](https://mlsanity.com/paper/1703.09179/) (2017)
- [Learning Music Representations with wav2vec 2.0](https://mlsanity.com/paper/2210.15310/) (2022)

## Built on by (most cited first, 22 of 22)

- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [Towards Holistic Evaluation of Large Audio-Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2505.15957/) (2025)
- [MARBLE: Music Audio Representation Benchmark for Universal Evaluation](https://mlsanity.com/paper/2306.10548/) (2023)
- [Foundation Models for Music: A Survey](https://mlsanity.com/paper/2408.14340/) (2024)
- [Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey](https://mlsanity.com/paper/2412.18619/) (2024)
- [MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Music](https://mlsanity.com/paper/2402.09871/) (2024)
- [LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPT](https://mlsanity.com/paper/2306.17103/) (2023)
- [Exploring Musical Roots: Applying Audio Embeddings to Empower Influence Attribution for a Generative Music Model](https://mlsanity.com/paper/2401.14542/) (2024)
- [On the Effectiveness of Speech Self-supervised Learning for Music](https://mlsanity.com/paper/2307.05161/) (2023)
- [MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning](https://mlsanity.com/paper/2310.09853/) (2023)
- [A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives](https://mlsanity.com/paper/2504.00837/) (2025)
- [Universal Music Representations? Evaluating Foundation Models on World Music Corpora](https://mlsanity.com/paper/2506.17055/) (2025)
- [Unsupervised Musical Object Discovery from Audio](https://mlsanity.com/paper/2311.07534/) (2023)
- [Toward Leveraging Pre-Trained Self-Supervised Frontends for Automatic Singing Voice Understanding Tasks: Three Case Studies](https://mlsanity.com/paper/2306.12714/) (2023)
- [Towards Training Music Taggers on Synthetic Data](https://mlsanity.com/paper/2407.02156/) (2024)
- [Real-world Music Plagiarism Detection With Music Segment Transcription System](https://mlsanity.com/paper/2509.08282/) (2025)
- [Segment Transformer: AI-Generated Music Detection via Music Structural Analysis](https://mlsanity.com/paper/2509.08283/) (2025)
- [Fusion Segment Transformer: Bi-Directional Attention Guided Fusion Network for AI-Generated Music Detection](https://mlsanity.com/paper/2601.13647/) (2026)
- [Are Music Foundation Models Better at Singing Voice Deepfake Detection? Far-Better Fuse them with Speech Foundation Models](https://mlsanity.com/paper/2409.14131/) (2024)
- [Auditing Training Data in Generative Music Models via Black-Box Membership Inference](https://mlsanity.com/paper/2605.29202/) (2026)
- [Multimodal Music Recommendation System using LLMs](https://mlsanity.com/paper/2606.00125/) (2026)
- [TrueMuse: A Benchmark for Data Attribution in Text-to-Music Models](https://mlsanity.com/paper/2610.00835/) (2026)

## Abstract (arXiv)

The deep learning community has witnessed an exponentially growing interest in self-supervised learning (SSL). However, it still remains unexplored how to build a framework for learning useful representations of raw music waveforms in a self-supervised manner. In this work, we design Music2Vec, a framework exploring different SSL algorithmic components and tricks for music audio recordings. Our model achieves comparable results to the state-of-the-art (SOTA) music SSL model Jukebox, despite being significantly smaller with less than 2% of parameters of the latter. The model will be released on Huggingface(Please refer to: https://huggingface.co/m-a-p/music2vec-v1)