# Crossing You in Style: Cross-modal Style Transfer from Music to Visual Arts

arXiv: https://arxiv.org/abs/2009.08083 · HTML: https://mlsanity.com/paper/2009.08083/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2020.

## Built on by (most cited first, 10 of 10)

- [Sound-Guided Semantic Image Manipulation](https://mlsanity.com/paper/2112.00007/) (2021)
- [Tr"aumerAI: Dreaming Music with StyleGAN](https://mlsanity.com/paper/2102.04680/) (2021)
- [AVI-Edit: Audio-sync Video Instance Editing with Granularity-Aware Mask Refiner](https://mlsanity.com/paper/2512.10571/) (2025)
- [Robust Sound-Guided Image Manipulation](https://mlsanity.com/paper/2208.14114/) (2022)
- [Generating Realistic Images from In-the-wild Sounds](https://mlsanity.com/paper/2309.02405/) (2023)
- [Network Bending of Diffusion Models for Audio-Visual Generation](https://mlsanity.com/paper/2406.19589/) (2024)
- [Cross-modal Face- and Voice-style Transfer](https://mlsanity.com/paper/2302.13838/) (2023)
- [Robot Synesthesia: A Sound and Emotion Guided AI Painter](https://mlsanity.com/paper/2302.04850/) (2023)
- [Combining Genre Classification and Harmonic-Percussive Features with Diffusion Models for Music-Video Generation](https://mlsanity.com/paper/2412.05694/) (2024)
- [Affect-aware Cross-Domain Recommendation for Art Therapy via Music Preference Elicitation](https://mlsanity.com/paper/2507.21120/) (2025)

## Abstract (arXiv)

Music-to-visual style transfer is a challenging yet important cross-modal learning problem in the practice of creativity. Its major difference from the traditional image style transfer problem is that the style information is provided by music rather than images. Assuming that musical features can be properly mapped to visual contents through semantic links between the two domains, we solve the music-to-visual style transfer problem in two steps: music visualization and style transfer. The music visualization network utilizes an encoder-generator architecture with a conditional generative adversarial network to generate image-based music representations from music data. This network is integrated with an image style transfer method to accomplish the style transfer process. Experiments are conducted on WikiArt-IMSLP, a newly compiled dataset including Western music recordings and paintings listed by decades. By utilizing such a label to learn the semantic connection between paintings and music, we demonstrate that the proposed framework can generate diverse image style representations from a music piece, and these representations can unveil certain art forms of the same era. Subjective testing results also emphasize the role of the era label in improving the perceptual quality on the compatibility between music and visual content.