# Remixing Music with Visual Conditioning

arXiv: https://arxiv.org/abs/2010.14565 · HTML: https://mlsanity.com/paper/2010.14565/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Oct 2020.

## Built on by (most cited first, 1 of 1)

- [InstructME: An Instruction Guided Music Edit And Remix Framework with Latent Diffusion Models](https://mlsanity.com/paper/2308.14360/) (2023)

## Abstract (arXiv)

We propose a visually conditioned music remixing system by incorporating deep visual and audio models. The method is based on a state of the art audio-visual source separation model which performs music instrument source separation with video information. We modified the model to work with user-selected images instead of videos as visual input during inference to enable separation of audio-only content. Furthermore, we propose a remixing engine that generalizes the task of source separation into music remixing. The proposed method is able to achieve improved audio quality compared to remixing performed by the separate-and-add method with a state-of-the-art audio-visual source separation model.