The artificial synesthete: Image-melody translations with variational autoencoders
Not on the map. Why?
Abstract (arXiv)
Abstract This project presents a system of neural networks to translate between images and melodies. Autoencoders compress the information in samples to abstract representation. A translation network learns a set of correspondences between musical and visual concepts from repeated joint exposure. The resulting "artificial synesthete" generates simple melodies inspired by images, and images from music. These are novel interpretation (not transposed data), expressing the machine' perception and understanding. Observing the work, one explores the machine's perception and thus, by contrast, one's own.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 48 references that are in the map. First posted on arXiv 6 Dec 2021.
Builds on
9 of 48 references.
References · 9
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2013Auto-Encoding Variational Bayescited by 4,053
- 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
- 2018A Hierarchical Latent Vector Model for Learning Long-Term Structure in Musiccited by 131
- 2016Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmarkcited by 71
- 2017Deep Learning Techniques for Music Generation -- A Surveycited by 49
- 2014AI Methods in Algorithmic Composition: A Comprehensive Surveycited by 12
- 2019Translating Visual Art into Musiccited by 2
- 2018Autoencoders for music sound modeling: a comparison of linear, shallow, deep, recurrent and variational modelscited by 1
Built on by
Not cited yet.