Decoupled Latent Flow Matching for Few-Step Joint Vocal-Accompaniment Separation
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
Generative modeling provides a flexible way to model mixture-conditioned source distributions, but iterative diffusion and flow matching models are costly for long music signals. This paper studies joint vocal-accompaniment separation through latent flow matching, where a pretrained variational autoencoder (VAE) maps mixtures and sources into a compact latent space and a flow matching model generates vocal and accompaniment latents jointly. The proposed framework decouples semantic separation from acoustic velocity prediction through a Separation Encoder and a Velocity Decoder. To reduce sampling cost, we further apply latent adversarial post-training inspired by Flow2GAN for few-step generation. Experiments show that latent adversarial refinement can improve perceptual and separation metrics under a reduced sampling budget.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 22 of its 25 references that are in the map. First posted on arXiv 31 Aug 2026.
Builds on
22 of 25 references.
Closest ancestors · 1
Same direction, newest first
Other references · 21
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2023Consistency Modelscited by 1,778
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
- 2016Least Squares Generative Adversarial Networkscited by 1,000
- 2023One-step Diffusion with Distribution Matching Distillationcited by 978
- 2023Adversarial Diffusion Distillationcited by 741
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
- 2025Mean Flows for One-step Generative Modelingcited by 524
9 more
- 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 400
- 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
- 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 175
- 2025Improved Mean Flows: On the Challenges of Fastforward Generative Modelscited by 102
- 2025DDT: Decoupled Diffusion Transformercited by 100
- 2021Hybrid Spectrogram and Waveform Source Separationcited by 90
- 2020ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metriccited by 76
- 2018The 2018 Signal Separation Evaluation Campaigncited by 41
- 2025Towards Reliable Objective Evaluation Metrics for Generative Singing Voice Separation Modelscited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Efficient and Fast Generative-Based Singing Voice Separation using a Latent Diffusion Model
- 2024FlowSep: Language-Queried Sound Separation with Rectified Flow Matching
- 2026FlowSep 2: Self-Supervised Flow Matching for Language-Queried Audio Source Separation
- 2026Mitigating Latent Mismatch in cVAE-Based Singing Voice Synthesis via Flow Matching
- 2025Source Separation by Flow Matching
- 2025High-Quality Sound Separation Across Diverse Categories via Visually-Guided Generative Modeling
- 2025Generating Separated Singing Vocals Using a Diffusion Model Conditioned on Music Mixtures
- 2026LaDA-Band: Language Diffusion Models for Vocal-to-Accompaniment Generation