# Cross-Modal Music-Video Recommendation: A Study of Design Choices

arXiv: https://arxiv.org/abs/2104.14799 · HTML: https://mlsanity.com/paper/2104.14799/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Apr 2021.

## Built on by (most cited first, 11 of 11)

- [MuseChat: A Conversational Music Recommendation System for Videos](https://mlsanity.com/paper/2310.06282/) (2023)
- [It's Time for Artistic Correspondence in Music and Video](https://mlsanity.com/paper/2206.07148/) (2022)
- [Language-Guided Music Recommendation for Video via Prompt Analogies](https://mlsanity.com/paper/2306.09327/) (2023)
- [FilmComposer: LLM-Driven Music Production for Silent Film Clips](https://mlsanity.com/paper/2503.08147/) (2025)
- [MOSA: Music Motion with Semantic Annotation Dataset for Cross-Modal Music Processing](https://mlsanity.com/paper/2406.06375/) (2024)
- [Bridging High-Quality Audio and Video via Language for Sound Effects Retrieval from Visual Queries](https://mlsanity.com/paper/2308.09089/) (2023)
- [A Survey of Recent Advances and Challenges in Deep Audio-Visual Correlation Learning](https://mlsanity.com/paper/2412.00049/) (2024)
- [Video-to-Music Recommendation using Temporal Alignment of Segments](https://mlsanity.com/paper/2306.07187/) (2023)
- [Start from Video-Music Retrieval: An Inter-Intra Modal Loss for Cross Modal Retrieval](https://mlsanity.com/paper/2407.19415/) (2024)
- [Recent Advances and Challenges in Deep Audio-Visual Correlation Learning](https://mlsanity.com/paper/2202.13673/) (2022)
- [Solving Copyright Infringement on Short Video Platforms: Novel Datasets and an Audio Restoration Deep Learning Pipeline](https://mlsanity.com/paper/2504.21772/) (2025)

## Abstract (arXiv)

In this work, we study music/video cross-modal recommendation, i.e. recommending a music track for a video or vice versa. We rely on a self-supervised learning paradigm to learn from a large amount of unlabelled data. We rely on a self-supervised learning paradigm to learn from a large amount of unlabelled data. More precisely, we jointly learn audio and video embeddings by using their co-occurrence in music-video clips. In this work, we build upon a recent video-music retrieval system (the VM-NET), which originally relies on an audio representation obtained by a set of statistics computed over handcrafted features. We demonstrate here that using audio representation learning such as the audio embeddings provided by the pre-trained MuSimNet, OpenL3, MusicCNN or by AudioSet, largely improves recommendations. We also validate the use of the cross-modal triplet loss originally proposed in the VM-NET compared to the binary cross-entropy loss commonly used in self-supervised learning. We perform all our experiments using the Music Video Dataset (MVD).