# MPJudge: Towards Perceptual Assessment of Music-Induced Paintings

arXiv: https://arxiv.org/abs/2511.07137 · HTML: https://mlsanity.com/paper/2511.07137/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 23 references that are in the map.
- First posted on arXiv 10 Nov 2025.

## Builds on (most cited first, 9 of 9)

- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Towards Artistic Image Aesthetics Assessment: a Large-scale Dataset and a New Method](https://mlsanity.com/paper/2303.15166/) (2023)
- [Coordinated Joint Multimodal Embeddings for Generalized Audio-Visual Zeroshot Classification and Retrieval of Videos](https://mlsanity.com/paper/1910.08732/) (2019)
- [Audio-visual Generalised Zero-shot Learning with Cross-modal Attention and Language](https://mlsanity.com/paper/2203.03598/) (2022)
- [AVGZSLNet: Audio-Visual Generalized Zero-Shot Learning by Reconstructing Label Features from Multi-Modal Embeddings](https://mlsanity.com/paper/2005.13402/) (2020)
- [Learning Affective Correspondence between Music and Image](https://mlsanity.com/paper/1904.00150/) (2019)
- [Emotion-Based End-to-End Matching Between Image and Music in Valence-Arousal Space](https://mlsanity.com/paper/2009.05103/) (2020)
- [Color: A Crucial Factor for Aesthetic Quality Assessment in a Subjective Dataset of Paintings](https://mlsanity.com/paper/1609.05583/) (2016)
- [PPJudge: Towards Human-Aligned Assessment of Artistic Painting Process](https://mlsanity.com/paper/2507.09242/) (2025)

## Abstract (arXiv)

Music induced painting is a unique artistic practice, where visual artworks are created under the influence of music. Evaluating whether a painting faithfully reflects the music that inspired it poses a challenging perceptual assessment task. Existing methods primarily rely on emotion recognition models to assess the similarity between music and painting, but such models introduce considerable noise and overlook broader perceptual cues beyond emotion. To address these limitations, we propose a novel framework for music induced painting assessment that directly models perceptual coherence between music and visual art. We introduce MPD, the first large scale dataset of music painting pairs annotated by domain experts based on perceptual coherence. To better handle ambiguous cases, we further collect pairwise preference annotations. Building on this dataset, we present MPJudge, a model that integrates music features into a visual encoder via a modulation based fusion mechanism. To effectively learn from ambiguous cases, we adopt Direct Preference Optimization for training. Extensive experiments demonstrate that our method outperforms existing approaches. Qualitative results further show that our model more accurately identifies music relevant regions in paintings.