mlsanityData to 7 Oct 2026

MMAudioSep: Taming Video-to-Audio Generative Model Towards Video/Text-Queried Sound Separation

arXiv 2510.09065 ↗First posted 10 Oct 2025cs.SD cs.CV cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

We introduce MMAudioSep, a generative model for video/text-queried sound separation that is founded on a pretrained video-to-audio model. By leveraging knowledge about the relationship between video/text and audio learned through a pretrained audio generative model, we can train the model more efficiently, i.e., the model does not need to be trained from scratch. We evaluate the performance of MMAudioSep by comparing it to existing separation models, including models based on both deterministic and generative approaches, and find it is superior to the baseline models. Furthermore, we demonstrate that even after acquiring functionality for sound separation via fine-tuning, the model retains the ability for original video-to-audio generation. This highlights the potential of foundational sound generation models to be adopted for sound-related downstream tasks. Our code is available at https://github.com/sony/mmaudiosep.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 33 of its 36 references that are in the map. First posted on arXiv 10 Oct 2025.

Builds on

33 of 36 references.

References · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  3. 2016Improved Techniques for Training GANscited by 2,938
  4. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  5. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  6. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  7. 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 641
  8. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 563
  9. 2019Clotho: An Audio Captioning Datasetcited by 354
  10. 2023WavCaps: A ChatGPT-Assisted Weakly-Labelled Audio Captioning Dataset for Audio-Language Multimodal Researchcited by 266
  11. 2018The Sound of Pixelscited by 258
  12. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 245
21 more
  1. 2021Efficient Training of Audio Transformers with Patchoutcited by 222
  2. 2018Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithmscited by 221
  3. 2019The Sound of Motionscited by 145
  4. 2024MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesiscited by 134
  5. 2024Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Alignerscited by 111
  6. 2019Co-Separating Sounds of Visual Objectscited by 110
  7. 2024FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Soundscited by 109
  8. 2024Synchformer: Efficient Synchronization from Sparse Cuescited by 94
  9. 2023V2A-Mapper: A Lightweight Solution for Vision-to-Audio Generation by Connecting Foundation Modelscited by 65
  10. 2024Frieren: Efficient Video-to-Audio Generation Network with Rectified Flow Matchingcited by 52
  11. 2024Temporally Aligned Audio for Video with Autoregressioncited by 51
  12. 2024Read, Watch and Scream! Sound Generation from Text and Videocited by 48
  13. 2022Separate What You Describe: Language-Queried Audio Source Separationcited by 45
  14. 2023Separate Anything You Describecited by 43
  15. 2022iQuery: Instruments as Queries for Audio-Visual Sound Separationcited by 29
  16. 2022CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videoscited by 23
  17. 2024FlowSep: Language-Queried Sound Separation with Rectified Flow Matchingcited by 21
  18. 2021Visually Guided Sound Source Separation and Localization using Self-Supervised Motion Representationscited by 18
  19. 2024OmniSep: Unified Omni-Modality Sound Separation with Query-Mixupcited by 8
  20. 202530+ Years of Source Separation Research: Achievements and Future Challengescited by 7
  21. 2025SpecMaskFoley: Steering Pretrained Spectral Masked Generative Transformer Toward Synchronized Video-to-audio Synthesis via ControlNetcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026MMAudioReverbs: Video-Guided Acoustic Modeling for Dereverberation and Room Impulse Response Estimationcited by 0
  2. 2026MMAudio-LABEL: Audio Event Labeling via Audio Generation for Silent Videocited by 0
  3. 2026Joint and Cross-Modal Video-Audio Generation and Editing: A Unified Formulation and Design Taxonomycited by 0