VQCPC-GAN: Variable-Length Adversarial Audio Synthesis Using Vector-Quantized Contrastive Predictive Coding
Not on the map. Why?
Abstract (arXiv)
Influenced by the field of Computer Vision, Generative Adversarial Networks (GANs) are often adopted for the audio domain using fixed-size two-dimensional spectrogram representations as the "image data". However, in the (musical) audio domain, it is often desired to generate output of variable duration. This paper presents VQCPC-GAN, an adversarial framework for synthesizing variable-length audio by exploiting Vector-Quantized Contrastive Predictive Coding (VQCPC). A sequence of VQCPC tokens extracted from real audio data serves as conditional input to a GAN architecture, providing step-wise time-dependent features of the generated content. The input noise z (characteristic in adversarial architectures) remains fixed over time, ensuring temporal consistency of global features. We evaluate the proposed model by comparing a diverse set of metrics against various strong baselines. Results show that, even though the baselines score best, VQCPC-GAN achieves comparable performance even when generating variable-length audio. Numerous sound examples are provided in the accompanying website, and we release the code for reproducibility.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 4 May 2021.
Builds on
28 of 34 references.
References · 28
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2017Neural Discrete Representation Learningcited by 4,650
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2016Improved Techniques for Training GANscited by 2,938
- 2017Improved Training of Wasserstein GANscited by 2,223
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
- 2018Demystifying MMD GANscited by 1,268
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
- 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 562
16 more
- 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
- 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
- 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
- 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 222
- 2018Adversarial Audio Synthesiscited by 222
- 2018Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithmscited by 221
- 2016C-RNN-GAN: Continuous recurrent neural networks with adversarial trainingcited by 132
- 2019GANSynth: Adversarial Neural Audio Synthesiscited by 114
- 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 90
- 2020Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challengecited by 55
- 2020DrumGAN: Synthesis of Drum Sounds With Timbral Feature Conditioning Using Generative Adversarial Networkscited by 34
- 2020Comparing Representations for Audio Synthesis Using Generative Adversarial Networkscited by 13
- 2020Audio Codec Enhancement with Generative Adversarial Networkscited by 11
- 2020Vector Quantized Contrastive Predictive Coding for Template-based Music Generationcited by 10
- 2018A Multi-Discriminator CycleGAN for Unsupervised Non-Parallel Speech Domain Adaptationcited by 9
- 2021Signal Representations for Synthesizing Audio Textures with Generative Adversarial Networkscited by 5
Built on by
Citing papers per year: 2022: 1; 2023: 0; 2024: 0; 2025: 1; 2026: 1.
3 papers.