mlsanityData to 7 Oct 2026

GANStrument: Adversarial Instrument Sound Synthesis with Pitch-invariant Instance Conditioning

arXiv 2211.05385 ↗First posted 10 Nov 2022cs.SD cs.LG eess.AScited by 9

Not on the map. Why?

Abstract (arXiv)

We propose GANStrument, a generative adversarial model for instrument sound synthesis. Given a one-shot sound as input, it is able to generate pitched instrument sounds that reflect the timbre of the input within an interactive time. By exploiting instance conditioning, GANStrument achieves better fidelity and diversity of synthesized sounds and generalization ability to various inputs. In addition, we introduce an adversarial training scheme for a pitch-invariant feature extractor that significantly improves the pitch accuracy and timbre consistency. Experimental results show that GANStrument outperforms strong baselines that do not use instance conditioning in terms of generation quality and input editability. Qualitative examples are available online.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 21 of its 26 references that are in the map. First posted on arXiv 10 Nov 2022.

Builds on

21 of 26 references.

References · 21

  1. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  2. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  3. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  5. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  6. 2021GAN Inversion: A Surveycited by 366
  7. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  8. 2018cGANs with Projection Discriminatorcited by 253
  9. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 245
  10. 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 222
  11. 2020DDSP: Differentiable Digital Signal Processingcited by 177
  12. 2021Instance-Conditioned GANcited by 119
9 more
  1. 2017Fader Networks: Manipulating Images by Sliding Attributescited by 118
  2. 2019GANSynth: Adversarial Neural Audio Synthesiscited by 114
  3. 2021TorchAudio: Building Blocks for Audio and Speech Processingcited by 83
  4. 2019MelNet: A Generative Model for Audio in the Frequency Domaincited by 41
  5. 2020DrumGAN: Synthesis of Drum Sounds With Timbral Feature Conditioning Using Generative Adversarial Networkscited by 34
  6. 2018A Universal Music Translation Networkcited by 22
  7. 2019Learning Disentangled Representations of Timbre and Pitch for Musical Instrument Sounds Using Gaussian Mixture Variational Autoencoderscited by 21
  8. 2022DDSP-based Singing Vocoders: A New Subtractive-based Synthesizer and A Comprehensive Evaluationcited by 11
  9. 2021Differentiable Wavetable Synthesiscited by 4

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 4; 2025: 3.

9 papers.

  1. 2024Expressive Acoustic Guitar Sound Synthesis with an Instrument-Specific Input Representation and Diffusion Outpaintingcited by 5
  2. 2025TokenSynth: A Token-based Neural Synthesizer for Instrument Cloning and Text-to-Instrumentcited by 5
  3. 2023InstrumentGen: Generating Sample-Based Musical Instruments From Textcited by 4
  4. 2023Annotation-free Automatic Music Transcription with Scalable Synthetic Data and Adversarial Domain Confusioncited by 2
  5. 2024HyperGANStrument: Instrument Sound Synthesis and Editing with Pitch-Invariant Hypernetworkscited by 1
  6. 2024Generating Sample-Based Musical Instruments Using Neural Audio Codec Language Modelscited by 1
  7. 2024Automatic Equalization for Individual Instrument Tracks Using Convolutional Neural Networkscited by 1
  8. 2025Designing Neural Synthesizers for Low-Latency Interactioncited by 1
  9. 2025Pitch-Conditioned Instrument Sound Synthesis From an Interactive Timbre Latent Spacecited by 0