mlsanityData to 14 Oct 2026

Schrodinger Audio-Visual Editor: Object-Level Audiovisual Removal

arXiv 2512.12875 ↗First posted 14 Dec 2025cs.CV cs.MM cs.SDcited by 4

Not on the map. Why?

Abstract (arXiv)

Joint editing of audio and visual content is crucial for precise and controllable content creation. This new task poses challenges due to the limitations of paired audio-visual data before and after targeted edits, and the heterogeneity across modalities. To address the data and modeling challenges in joint audio-visual editing, we introduce SAVEBench, a paired audiovisual dataset with text and mask conditions to enable object-grounded source-to-target learning. With SAVEBench, we train the Schrodinger Audio-Visual Editor (SAVE), an end-to-end flow-matching model that edits audio and video in parallel while keeping them aligned throughout processing. SAVE incorporates a Schrodinger Bridge that learns a direct transport from source to target audiovisual mixtures. Our evaluation demonstrates that the proposed SAVE model is able to remove the target objects in audio and visual content while preserving the remaining content, with stronger temporal synchronization and audiovisual semantic correspondence compared with pairwise combinations of an audio editor and a video editor.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 61 of its 74 references that are in the map. First posted on arXiv 14 Dec 2025.

Builds on

61 of 74 references.

References · 61

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  7. 2022Scalable Diffusion Models with Transformerscited by 6,054
  8. 2022Flow Matching for Generative Modelingcited by 5,702
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  12. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
38 more
  1. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  2. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  3. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  4. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 879
  5. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
  6. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  7. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modelingcited by 602
  8. 2018Video-to-Video Synthesiscited by 390
  9. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  10. 2023Inpaint Anything: Segment Anything Meets Image Inpaintingcited by 233
  11. 2021Likelihood Training of Schr"odinger Bridge using Forward-Backward SDEs Theorycited by 219
  12. 2023Any-to-Any Generation via Composable Diffusioncited by 204
  13. 2023ProPainter: Improving Propagation and Transformer for Video Inpaintingcited by 197
  14. 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 155
  15. 2021Taming Visually Guided Sound Generationcited by 136
  16. 2024MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesiscited by 134
  17. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  18. 2014On the relation between optimal transport and Schr"odinger bridges: A stochastic control viewpointcited by 99
  19. 2024Synchformer: Efficient Synchronization from Sparse Cuescited by 97
  20. 2024RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Modelscited by 89
  21. 2025VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Controlcited by 84
  22. 2023AUDIT: Audio Editing by Following Instructions with Latent Diffusion Modelscited by 83
  23. 2022LVOS: A Benchmark for Long-term Video Object Segmentationcited by 77
  24. 2024CV-VAE: A Compatible Video VAE for Latent Generative Video Modelscited by 70
  25. 2023Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptationcited by 69
  26. 2023V2A-Mapper: A Lightweight Solution for Vision-to-Audio Generation by Connecting Foundation Modelscited by 65
  27. 2024Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversioncited by 60
  28. 2021ACAV100M: Automatic Curation of Large-Scale Datasets for Audio-Visual Video Representation Learningcited by 50
  29. 2024Read, Watch and Scream! Sound Generation from Text and Videocited by 48
  30. 2023Sound to Visual Scene Generation by Audio-to-Visual Latent Alignmentcited by 41
  31. 2023The Power of Sound (TPoS): Audio Reactive Video Generation with Stable Diffusioncited by 41
  32. 2025Kling-Foley: Multimodal Diffusion Transformer for High-Quality Video-to-Audio Generationcited by 40
  33. 2024Paint by Inpaint: Learning to Add Image Objects by Removing Them Firstcited by 39
  34. 2024Towards Language-Driven Video Inpainting via Multimodal Large Language Modelscited by 33
  35. 2024Audio-Synchronized Visual Animationcited by 32
  36. 2024Masked Generative Video-to-Audio Transformers with Enhanced Synchronicitycited by 28
  37. 2024SyncFlow: Toward Temporally Aligned Joint Audio-Video Generation from Textcited by 26
  38. 2023SyncFusion: Multimodal Onset-synchronized Video-to-Audio Foley Synthesiscited by 24

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026InstructAV2AV: Instruction-Guided Audio-Video Joint Editingcited by 5
  2. 2026Audio-Visual Intelligence in Large Foundation Modelscited by 2
  3. 2026AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilitiescited by 1
  4. 2026TV-AudioRemover: Joint Text-Visual Guided Sound Removal with Multi-Task Hard-Mixture Curriculumcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AV-FlowEdit: Training-Free Prompt-Driven Audio-Visual Editing with Event-Aware Spectral Trajectory Preservation
  2. 2025AV-Edit: Multimodal Generative Sound Effect Editing via Audio-Visual Semantic Joint Control
  3. 2026JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curation
  4. 2025AudioMorphix: Training-free audio editing with diffusion probabilistic models
  5. 2026Soundwich: Video Generation with Layered and Controllable Audio
  6. 2025Audio-Guided Visual Editing with Complex Multi-Modal Prompts
  7. 2024SoundBrush: Sound as a Brush for Visual Scene Editing
  8. 2025Guiding Audio Editing with Audio Language Model