mlsanityData to 7 Oct 2026

AVI-Edit: Audio-sync Video Instance Editing with Granularity-Aware Mask Refiner

arXiv 2512.10571 ↗First posted 11 Dec 2025cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Recent advancements in video generation highlight that realistic audio-visual synchronization is crucial for engaging content creation. However, existing video editing methods largely overlook audio-visual synchronization and lack the fine-grained spatial and temporal controllability required for precise instance-level edits. In this paper, we propose AVI-Edit, a framework for audio-sync video instance editing. We propose a granularity-aware mask refiner that iteratively refines coarse user-provided masks into precise instance-level regions. We further design a self-feedback audio agent to curate high-quality audio guidance, providing fine-grained temporal control. To facilitate this task, we additionally construct a large-scale dataset with instance-centric correspondence and comprehensive annotations. Extensive experiments demonstrate that AVI-Edit outperforms state-of-the-art methods in visual quality, condition following, and audio-visual synchronization. Project page: https://hjzheng.net/projects/AVI-Edit/.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 55 of its 67 references that are in the map. First posted on arXiv 11 Dec 2025.

Builds on

55 of 67 references.

References · 55

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2025Qwen2.5-VL Technical Reportcited by 5,418
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  8. 2016Improved Techniques for Training GANscited by 2,938
  9. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  10. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  11. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  12. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
38 more
  1. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  2. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  3. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  4. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
  5. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  6. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  7. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  8. 2024Genie: Generative Interactive Environmentscited by 706
  9. 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 641
  10. 2025VACE: All-in-One Video Creation and Editingcited by 467
  11. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  12. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  13. 2025Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Soundcited by 129
  14. 2023RAVE: Randomized Noise Shuffling for Fast and Consistent Video Editing with Diffusion Modelscited by 113
  15. 2023VidToMe: Video Token Merging for Zero-Shot Video Editingcited by 111
  16. 2020Condensed Movies: Story Based Retrieval with Contextual Embeddingscited by 95
  17. 2025VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Controlcited by 83
  18. 2025Ovi: Twin Backbone Cross-Modal Fusion for Audio-Video Generationcited by 76
  19. 2024Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversioncited by 58
  20. 2025UniVerse-1: Unified Audio-Video Generation via Stitching of Expertscited by 58
  21. 2019Speech2Face: Learning the Face Behind a Voicecited by 55
  22. 2025HunyuanVideo-Foley: Multimodal Diffusion with Representation Alignment for High-Fidelity Foley Audio Generationcited by 42
  23. 2024VideoCLIP-XL: Advancing Long Description Understanding for Video CLIP Modelscited by 40
  24. 2022Sound-Guided Semantic Video Generationcited by 35
  25. 2024MovieBench: A Hierarchical Movie Level Dataset for Long Video Generationcited by 35
  26. 2025ContextFlow: Training-Free Video Object Editing via Adaptive Context Enrichmentcited by 27
  27. 2023AudioToken: Adaptation of Text-Conditioned Diffusion Models for Audio-to-Image Generationcited by 22
  28. 2025InterActHuman: Multi-Concept Human Animation with Layout-Aligned Audio Conditionscited by 20
  29. 2025MIDAS: Multimodal Interactive Digital-humAn Synthesis via Real-time Autoregressive Video Generationcited by 18
  30. 2025Generative AI for Film Creation: A Survey of Recent Advancescited by 17
  31. 2023Soundini: Sound-Guided Diffusion for Natural Video Editingcited by 14
  32. 2025Audio-Sync Video Generation with Multi-Stream Temporal Controlcited by 14
  33. 2025Zero-Shot Audio-Visual Editing via Cross-Modal Delta Denoisingcited by 13
  34. 2025Object-AVEdit: An Object-level Audio-Visual Editing Modelcited by 12
  35. 2021Tr"aumerAI: Dreaming Music with StyleGANcited by 11
  36. 2023The Sound Demixing Challenge 2023 unicodex2013 Cinematic Demixing Trackcited by 11
  37. 2020Crossing You in Style: Cross-modal Style Transfer from Music to Visual Artscited by 10
  38. 2025O-DisCo-Edit: Object Distortion Control for Unified Realistic Video Editingcited by 10

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

  1. 2026SpongeBob: Sync-Aware Harmonious Audio-Visual Generative Editingcited by 6
  2. 2026InstructAV2AV: Instruction-Guided Audio-Video Joint Editingcited by 4
  3. 2026JAVEDIT: Joint Audio-Visual Instruction-Guided Video Editing with Agentic Data Curationcited by 1
  4. 2026AVE-Compass: Towards Holistic Evaluation for Audio-Video Editing Abilitiescited by 1
  5. 2025JoVA: Unified Multimodal Learning for Joint Video-Audio Generation and Editingcited by 0
  6. 2026MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Controlcited by 0
  7. 2026AVENUE: Audio-Video EditiNg Understanding and Evaluationcited by 0
  8. 2026AVIO: Learning to Add and Remove Sounding Objects in Audiovisual Scenescited by 0