# Towards Audio to Scene Image Synthesis using Generative Adversarial Network

arXiv: https://arxiv.org/abs/1808.04108 · HTML: https://mlsanity.com/paper/1808.04108/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2018.

## Built on by (most cited first, 23 of 23)

- [Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptation](https://mlsanity.com/paper/2309.16429/) (2023)
- [Multimodal Image Synthesis and Editing: The Generative AI Era](https://mlsanity.com/paper/2112.13592/) (2021)
- [Sound to Visual Scene Generation by Audio-to-Visual Latent Alignment](https://mlsanity.com/paper/2303.17490/) (2023)
- [RepMix: Representation Mixing for Robust Attribution of Synthesized Images](https://mlsanity.com/paper/2207.02063/) (2022)
- [Sound-Guided Semantic Image Manipulation](https://mlsanity.com/paper/2112.00007/) (2021)
- [A Review on Methods and Applications in Multimodal Deep Learning](https://mlsanity.com/paper/2202.09195/) (2022)
- [AudioToken: Adaptation of Text-Conditioned Diffusion Models for Audio-to-Image Generation](https://mlsanity.com/paper/2305.13050/) (2023)
- [Robust Sound-Guided Image Manipulation](https://mlsanity.com/paper/2208.14114/) (2022)
- [Collaborative Learning to Generate Audio-Video Jointly](https://mlsanity.com/paper/2104.02656/) (2021)
- [Visual-Tactile Cross-Modal Data Generation using Residue-Fusion GAN with Feature-Matching and Perceptual Losses](https://mlsanity.com/paper/2107.05468/) (2021)
- [Sound2Vision: Generating Diverse Visuals from Audio through Cross-Modal Latent Alignment](https://mlsanity.com/paper/2412.06209/) (2024)
- [Generating Realistic Images from In-the-wild Sounds](https://mlsanity.com/paper/2309.02405/) (2023)
- [The Boombox: Visual Reconstruction from Acoustic Vibrations](https://mlsanity.com/paper/2105.08052/) (2021)
- [MACS: Multi-source Audio-to-image Generation with Contextual Significance and Semantic Alignment](https://mlsanity.com/paper/2503.10287/) (2025)
- [The applicability of transperceptual and deep learning approaches to the study and mimicry of complex cartilaginous tissues](https://mlsanity.com/paper/2211.14314/) (2022)
- [Exploring Efficient-Tuned Learning Audio Representation Method from BriVL](https://mlsanity.com/paper/2303.04585/) (2023)
- [UniBriVL: Robust Universal Representation and Generation of Audio Driven Diffusion Models](https://mlsanity.com/paper/2307.15898/) (2023)
- [Designing ReLU Generative Networks to Enumerate Trees with a Given Tree Edit Distance](https://mlsanity.com/paper/2510.10706/) (2025)
- [Towards Unseen Triples: Effective Text-Image-joint Learning for Scene Graph Generation](https://mlsanity.com/paper/2306.13420/) (2023)
- [Deep Video Inpainting Guided by Audio-Visual Self-Supervision](https://mlsanity.com/paper/2310.07663/) (2023)
- [SeeingSounds: Learning Audio-to-Visual Alignment via Text](https://mlsanity.com/paper/2510.11738/) (2025)
- [ReLU Networks for Exact Generation of Similar Graphs](https://mlsanity.com/paper/2604.05929/) (2026)
- [Face and Voice Cross-modal Association with Learning Convex Feature Embedding](https://mlsanity.com/paper/2607.28129/) (2026)

## Abstract (arXiv)

Humans can imagine a scene from a sound. We want machines to do so by using conditional generative adversarial networks (GANs). By applying the techniques including spectral norm, projection discriminator and auxiliary classifier, compared with naive conditional GAN, the model can generate images with better quality in terms of both subjective and objective evaluations. Almost three-fourth of people agree that our model have the ability to generate images related to sounds. By inputting different volumes of the same sound, our model output different scales of changes based on the volumes, showing that our model truly knows the relationship between sounds and images to some extent.