InstructVEdit: A Holistic Approach for Instructional Video Editing
Not on the map. Why?
Abstract (arXiv)
Video editing according to instructions is a highly challenging task due to the difficulty in collecting large-scale, high-quality edited video pair data. This scarcity not only limits the availability of training data but also hinders the systematic exploration of model architectures and training strategies. While prior work has improved specific aspects of video editing (e.g., synthesizing a video dataset using image editing techniques or decomposed video editing training), a holistic framework addressing the above challenges remains underexplored. In this study, we introduce InstructVEdit, a full-cycle instructional video editing approach that: (1) establishes a reliable dataset curation workflow to initialize training, (2) incorporates two model architectural improvements to enhance edit quality while preserving temporal consistency, and (3) proposes an iterative refinement strategy leveraging real-world data to enhance generalization and minimize train-test discrepancies. Extensive experiments show that InstructVEdit achieves state-of-the-art performance in instruction-based video editing, demonstrating robust adaptability to diverse real-world scenarios. Project page: https://o937-blip.github.io/InstructVEdit.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 42 of its 43 references that are in the map. First posted on arXiv 22 Mar 2025.
Builds on
42 of 43 references.
References · 42
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
- 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
30 more
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
- 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
- 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
- 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
- 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 666
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
- 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
- 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
- 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
- 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
- 2023Pix2Video: Video Editing using Image Diffusioncited by 318
- 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
- 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
- 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
- 2023MOSE: A New Dataset for Video Object Segmentation in Complex Scenescited by 232
- 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
- 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 191
- 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 151
- 2023Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Modelscited by 135
- 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
- 2023RAVE: Randomized Noise Shuffling for Fast and Consistent Video Editing with Diffusion Modelscited by 113
- 2023Kosmos-G: Generating Images in Context with Multimodal Large Language Modelscited by 105
- 2023Consistent Video-to-Video Transfer Using Synthetic Datasetcited by 79
- 2022LVOS: A Benchmark for Long-term Video Object Segmentationcited by 76
- 2023Edit-A-Video: Single Video Editing with Object-Aware Consistencycited by 65
- 2024I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Modelscited by 55
- 2024Video Editing via Factorized Diffusion Distillationcited by 40
- 2023Fairy: Fast Parallelized Instruction-Guided Video-to-Video Synthesiscited by 39
- 2020Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructionscited by 25
Built on by
Citing papers per year: 2025: 3; 2026: 4.
7 papers.
- 2025Scaling Instruction-Based Video Editing with a High-Quality Synthetic Datasetcited by 76
- 2025Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformerscited by 34
- 2025In-Context Learning with Unpaired Clips for Instruction-based Video Editingcited by 34
- 2026Instruction-Based Video Editing by Repurposing an Image Editing Modelcited by 2
- 2026GRNEdit: Efficient General Video Editing from a New Binary-Evidence Perspective in Generative Refinement Networkscited by 2
- 2026OTPrune: Distribution-Aligned Visual Token Pruning via Optimal Transportcited by 1
- 2026LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editingcited by 1
- 2026GRNEdit: Efficient General Video Editing from a New Binary-Evidence Perspective in Generative Refinement Networks17 Aug 2026
- 2026Instruction-Based Video Editing by Repurposing an Image Editing Model14 Aug 2026
- 2026LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing18 Apr 2026
- 2026OTPrune: Distribution-Aligned Visual Token Pruning via Optimal Transport22 Feb 2026
- 2025Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset17 Oct 2025
- 2025In-Context Learning with Unpaired Clips for Instruction-based Video Editing16 Oct 2025
- 2025Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers3 Jun 2025