# Learning Affective Correspondence between Music and Image

arXiv: https://arxiv.org/abs/1904.00150 · HTML: https://mlsanity.com/paper/1904.00150/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Mar 2019.

## Built on by (most cited first, 8 of 8)

- [A Comprehensive Survey on Video Saliency Detection with Auditory Information: the Audio-visual Consistency Perceptual is the Key!](https://mlsanity.com/paper/2206.13390/) (2022)
- [Robustness of Fusion-based Multimodal Classifiers to Cross-Modal Content Dilutions](https://mlsanity.com/paper/2211.02646/) (2022)
- [Vision-to-Music Generation: A Survey](https://mlsanity.com/paper/2503.21254/) (2025)
- [InfoMAE: Pair-Efficient Cross-Modal Alignment for Multimodal Time-Series Sensing Signals](https://mlsanity.com/paper/2504.09707/) (2025)
- [Emotion-Guided Image to Music Generation](https://mlsanity.com/paper/2410.22299/) (2024)
- [Analyzing Images for Music Recommendation](https://mlsanity.com/paper/2105.07135/) (2021)
- [MMVA: Multimodal Matching Based on Valence and Arousal across Images, Music, and Musical Captions](https://mlsanity.com/paper/2501.01094/) (2025)
- [MPJudge: Towards Perceptual Assessment of Music-Induced Paintings](https://mlsanity.com/paper/2511.07137/) (2025)

## Abstract (arXiv)

We introduce the problem of learning affective correspondence between audio (music) and visual data (images). For this task, a music clip and an image are considered similar (having true correspondence) if they have similar emotion content. In order to estimate this crossmodal, emotion-centric similarity, we propose a deep neural network architecture that learns to project the data from the two modalities to a common representation space, and performs a binary classification task of predicting the affective correspondence (true or false). To facilitate the current study, we construct a large scale database containing more than 3,500 music clips and 85,000 images with three emotion classes (positive, neutral, negative). The proposed approach achieves 61.67% accuracy for the affective correspondence prediction task on this database, outperforming two relevant and competitive baselines. We also demonstrate that our network learns modality-specific representations of emotion (without explicitly being trained with emotion labels), which are useful for emotion recognition in individual modalities.