# Vector Quantized Contrastive Predictive Coding for Template-based Music Generation

arXiv: https://arxiv.org/abs/2004.10120 · HTML: https://mlsanity.com/paper/2004.10120/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2020.

## Built on by (most cited first, 10 of 10)

- [Variable-rate discrete representation learning](https://mlsanity.com/paper/2103.06089/) (2021)
- [The Piano Inpainting Application](https://mlsanity.com/paper/2107.05944/) (2021)
- [ERNIE-Music: Text-to-Waveform Music Generation with Diffusion Models](https://mlsanity.com/paper/2302.04456/) (2023)
- [Learning long-term music representations via hierarchical contextual constraints](https://mlsanity.com/paper/2202.06180/) (2022)
- [VQCPC-GAN: Variable-Length Adversarial Audio Synthesis Using Vector-Quantized Contrastive Predictive Coding](https://mlsanity.com/paper/2105.01531/) (2021)
- [VQ-Style: Disentangling Style and Content in Motion with Residual Quantized Representations](https://mlsanity.com/paper/2602.02334/) (2026)
- [Music Proofreading with RefinPaint: Where and How to Modify Compositions given Context](https://mlsanity.com/paper/2407.09099/) (2024)
- [Spectrogram Inpainting for Interactive Generation of Instrument Sounds](https://mlsanity.com/paper/2104.07519/) (2021)
- [Relating Human Perception of Musicality to Prediction in a Predictive Coding Model](https://mlsanity.com/paper/2210.16587/) (2022)
- [Pushing Toward the Simplex Vertices: A Simple Remedy for Code Collapse in Smoothed Vector Quantization](https://mlsanity.com/paper/2509.22161/) (2025)

## Abstract (arXiv)

In this work, we propose a flexible method for generating variations of discrete sequences in which tokens can be grouped into basic units, like sentences in a text or bars in music. More precisely, given a template sequence, we aim at producing novel sequences sharing perceptible similarities with the original template without relying on any annotation; so our problem of generating variations is intimately linked to the problem of learning relevant high-level representations without supervision. Our contribution is two-fold: First, we propose a self-supervised encoding technique, named Vector Quantized Contrastive Predictive Coding which allows to learn a meaningful assignment of the basic units over a discrete set of codes, together with mechanisms allowing to control the information content of these learnt discrete representations. Secondly, we show how these compressed representations can be used to generate variations of a template sequence by using an appropriate attention pattern in the Transformer architecture. We illustrate our approach on the corpus of J.S. Bach chorales where we discuss the musical meaning of the learnt discrete codes and show that our proposed method allows to generate coherent and high-quality variations of a given template.