mlsanityData to 14 Oct 2026

Spot, Separate, and Enhance: Fully Generative Approach for Audio Mixing

arXiv 2609.29169 ↗First posted 24 Sep 2026cs.SD cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce Spot, Separate, and Enhance (SSE), the first multimodal, user-guided generative model for audio remixing and enhancement. SSE enhances video content by rebalancing the audio, removing unwanted audio sources, and reducing reverberation, guided by both video and textual descriptions. To support its training and evaluation, we propose DegradedMix, a new dataset built on the audio remixing benchmark MuddyMix. We also adopt evaluation metrics from generative modeling, which better capture the creative nature of remixing than standard reconstruction-based metrics. SSE outperforms existing baselines in both controllability and remixing quality, as shown by extensive experiments. Project page: https://sse-ai.notion.site

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 38 references that are in the map. First posted on arXiv 24 Sep 2026.

Builds on

33 of 38 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2016Improved Techniques for Training GANscited by 2,970
  7. 2023Consistency Modelscited by 1,822
  8. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  9. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
  10. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
  11. 2024Movie Gen: A Cast of Media Foundation Modelscited by 644
  12. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
21 more
  1. 2023Improved Techniques for Training Consistency Modelscited by 439
  2. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  3. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 344
  4. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
  5. 2021Efficient Training of Audio Transformers with Patchoutcited by 226
  6. 2018Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithmscited by 225
  7. 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 156
  8. 2022Hybrid Transformers for Music Source Separationcited by 134
  9. 2024MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesiscited by 134
  10. 2023Audiobox: Unified Audio Generation with Natural Language Promptscited by 120
  11. 2024Synchformer: Efficient Synchronization from Sparse Cuescited by 97
  12. 2020Condensed Movies: Story Based Retrieval with Contextual Embeddingscited by 95
  13. 2024Flash Diffusion: Accelerating Any Conditional Diffusion Model for Few Steps Image Generationcited by 56
  14. 2024Temporally Aligned Audio for Video with Autoregressioncited by 52
  15. 2023AV-NeRF: Learning Neural Fields for Real-World Audio-Visual Scene Synthesiscited by 49
  16. 2021Visually Informed Binaural Audio Generation without Binaural Audioscited by 46
  17. 2025LBM: Latent Bridge Matching for Fast Image-to-Image Translationcited by 36
  18. 2025SAM Audio: Segment Anything in Audiocited by 30
  19. 2024Remastering Divide and Remaster: A Cinematic Audio Source Separation Dataset with Multilingual Supportcited by 5
  20. 2025Learning to Highlight Audio by Watching Moviescited by 5
  21. 2026Conditional Flow Matching for Visually-Guided Acoustic Highlightingcited by 1

Built on by

Not cited yet.