# Signal Representations for Synthesizing Audio Textures with Generative Adversarial Networks

arXiv: https://arxiv.org/abs/2103.07390 · HTML: https://mlsanity.com/paper/2103.07390/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2021.

## Built on by (most cited first, 6 of 6)

- [DarkGAN: Exploiting Knowledge Distillation for Comprehensible Audio Synthesis with GANs](https://mlsanity.com/paper/2108.01216/) (2021)
- [Towards Controllable Audio Texture Morphing](https://mlsanity.com/paper/2304.11648/) (2023)
- [VQCPC-GAN: Variable-Length Adversarial Audio Synthesis Using Vector-Quantized Contrastive Predictive Coding](https://mlsanity.com/paper/2105.01531/) (2021)
- [Example-Based Framework for Perceptually Guided Audio Texture Generation](https://mlsanity.com/paper/2308.11859/) (2023)
- [SpecSinGAN: Sound Effect Variation Synthesis Using Single-Image GANs](https://mlsanity.com/paper/2110.07311/) (2021)
- [Pitch-Conditioned Instrument Sound Synthesis From an Interactive Timbre Latent Space](https://mlsanity.com/paper/2510.04339/) (2025)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) currently achieve the state-of-the-art sound synthesis quality for pitched musical instruments using a 2-channel spectrogram representation consisting of log magnitude and instantaneous frequency (the "IFSpectrogram"). Many other synthesis systems use representations derived from the magnitude spectra, and then depend on a backend component to invert the output magnitude spectrograms that generally result in audible artefacts associated with the inversion process. However, for signals that have closely-spaced frequency components such as non-pitched and other noisy sounds, training the GAN on the 2-channel IFSpectrogram representation offers no advantage over the magnitude spectra based representations. In this paper, we propose that training GANs on single-channel magnitude spectra, and using the Phase Gradient Heap Integration (PGHI) inversion algorithm is a better comprehensive approach for audio synthesis modeling of diverse signals that include pitched, non-pitched, and dynamically complex sounds. We show that this method produces higher-quality output for wideband and noisy sounds, such as pops and chirps, compared to using the IFSpectrogram. Furthermore, the sound quality for pitched sounds is comparable to using the IFSpectrogram, even while using a simpler representation with half the memory requirements.