# Cinematic Audio Source Separation Using Visual Cues

arXiv: https://arxiv.org/abs/2603.26113 · HTML: https://mlsanity.com/paper/2603.26113/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2026.

## Built on by (most cited first, 1 of 1)

- [SepGen: Multi-Stem Audio-Video Separation and Generation in a Single Model](https://mlsanity.com/paper/2610.11361/) (2026)

## Abstract (arXiv)

Cinematic Audio Source Separation (CASS) aims to decompose mixed film audio into speech, music, and sound effects, enabling applications like dubbing and remastering. Existing CASS approaches are audio-only, overlooking the inherent audio-visual nature of films, where sounds often align with visual cues. We present the first framework for audio-visual CASS (AV-CASS), leveraging visual context to enhance separation quality. Our method formulates CASS as a conditional generative modeling problem using conditional flow matching, enabling multimodal audio source separation. To address the lack of cinematic datasets with isolated sound tracks, we introduce a training data synthesis pipeline that pairs in-the-wild audio and video streams (e.g., facial videos for speech, scene videos for effects) and design a dedicated visual encoder for this dual-stream setup. Trained entirely on synthetic data, our model generalizes effectively to real-world cinematic content and achieves strong performance on synthetic, real-world, and audio-only CASS benchmarks. Code and demo are available at urlhttps://cass-flowmatching.github.io.