mlsanityData to 7 Oct 2026

COCOLA: Coherence-Oriented Contrastive Learning of Musical Audio Representations

arXiv 2404.16969 ↗First posted 25 Apr 2024cs.SD cs.LG eess.AScited by 14

Not on the map. Why?

Abstract (arXiv)

We present COCOLA (Coherence-Oriented Contrastive Learning for Audio), a contrastive learning method for musical audio representations that captures the harmonic and rhythmic coherence between samples. Our method operates at the level of the stems composing music tracks and can input features obtained via Harmonic-Percussive Separation (HPS). COCOLA allows the objective evaluation of generative models for music accompaniment generation, which are difficult to benchmark with established metrics. In this regard, we evaluate recent music accompaniment generation models, demonstrating the effectiveness of the proposed method. We release the model checkpoints trained on public datasets containing separate stems (MUSDB18-HQ, MoisesDB, Slakh2100, and CocoChorales).

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 62 of its 85 references that are in the map. First posted on arXiv 25 Apr 2024.

Builds on

62 of 85 references.

References · 62

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  8. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  10. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
38 more
  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2022Classifier-Free Diffusion Guidancecited by 5,717
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  6. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  7. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  8. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  9. 2022High Fidelity Neural Audio Compressioncited by 673
  10. 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 641
  11. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
  12. 2023Simple and Controllable Music Generationcited by 480
  13. 2023MusicLM: Generating Music From Textcited by 442
  14. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  15. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 343
  16. 2022Masked Autoencoders that Listencited by 271
  17. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 266
  18. 2016FMA: A Dataset For Music Analysiscited by 205
  19. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 178
  20. 2020Contrastive Learning of General-Purpose Audio Representationscited by 168
  21. 2023Adapting Frechet Audio Distance for Generative Music Evaluationcited by 92
  22. 2019Learning Problem-agnostic Speech Representations from Multiple Self-supervised Taskscited by 87
  23. 2019Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantitycited by 86
  24. 2022MuLan: A Joint Embedding of Music Audio and Natural Languagecited by 85
  25. 2021Contrastive Learning of Musical Representationscited by 81
  26. 2023AUDIT: Audio Editing by Following Instructions with Latent Diffusion Modelscited by 81
  27. 2023Mousai: Text-to-Music Generation with Long-Context Latent Diffusioncited by 77
  28. 2024ChatMusician: Understanding and Generating Music Intrinsically with LLMcited by 67
  29. 2023Multi-Source Diffusion Models for Simultaneous Music Generation and Separationcited by 63
  30. 2023VampNet: Music Generation via Masked Acoustic Token Modelingcited by 57
  31. 2023SingSong: Generating musical accompaniments from singingcited by 54
  32. 2017Unsupervised Learning of Semantic Audio Representationscited by 44
  33. 2023Moisesdb: A dataset for source separation beyond 4-stemscited by 37
  34. 2020Source Separation with Deep Generative Priorscited by 35
  35. 2022Contrastive Audio-Language Learning for Musiccited by 32
  36. 2023StemGen: A music generation model that listenscited by 32
  37. 2020CLAR: Contrastive Learning of Auditory Representationscited by 31
  38. 2021MIDI-DDSP: Detailed Control of Musical Performance via Hierarchical Modelingcited by 31

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 3; 2026: 7.

14 papers.

Sort:

By area:

  1. 2024Foundation Models for Music: A Surveycited by 34
  2. 2025Aligning Text-to-Music Evaluation with Human Preferencescited by 20
  3. 2024Latent Diffusion Bridges for Unsupervised Musical Audio Timbre Transfercited by 9
  4. 2024Naturalistic Music Decoding from EEG Data via Latent Diffusion Modelscited by 8
  5. 2024High-Resolution Speech Restoration with Latent Diffusion Modelcited by 8
  6. 2025SpecMaskFoley: Steering Pretrained Spectral Masked Generative Transformer Toward Synchronized Video-to-audio Synthesis via ControlNetcited by 5
  7. 2025MGE-LDM: Joint Latent Diffusion for Simultaneous Music Generation and Source Extractioncited by 4
  8. 2026DARC: Drum accompaniment generation with fine-grained rhythm controlcited by 2
  9. 2026V2M-Zero: Zero-Pair Time-Aligned Video-to-Music Generationcited by 2
  10. 2026Towards Real-Time Human-AI Musical Co-Performance: Accompaniment Generation with Latent Diffusion Models and MAX/MSPcited by 2
  11. 2026Stemphonic: All-at-once Flexible Multi-stem Music Generationcited by 1
  12. 2026PHALAR: Phasors for Learned Musical Audio Representationscited by 1
2 more
  1. 2026Live Music Diffusion Models: Efficient Fine-Tuning and Post-Training of Interactive Diffusion Music Generatorscited by 1
  2. 2026JazzSAMBA: A Synchronous and Asynchronous Multi-take Band Audio Dataset of Jazz Standards for Live Music Modelscited by 0