ReBind: Multi-Reference Video Editing via Structured Instructions with Explicit Reference Relationships
On the map: Diffusion models → Video generation
Abstract (arXiv)
Recent diffusion-based video generation models have made significant progress in multi-reference image-conditioned video editing. However, existing methods still struggle to coordinate information from multiple visual sources accurately. We identify a critical deficiency in existing approaches. Existing editing instructions lack explicit reference relationships, and most multimodal large language models (MLLMs) cannot generate them reliably. To address this problem, we propose ReBind, a systematic framework that introduces semantic instructions with embedded reference tokens as the intermediate representation for multi-reference image-conditioned video editing. Our key insight is embedding reference tokens at semantic positions to eliminate ambiguity and establish precise bindings between visual attributes and their sources. We develop ReBind-Instruct, a specialized MLLM that learns to establish explicit bindings between visual attributes and their reference sources through a two-stage progressive scheme for precise reference relationships. We further develop ReBind-Edit, which enables lightweight adaptation of text-to-video models to coordinate multiple references by binding visual attributes to their designated sources. Extensive experiments demonstrate that ReBind substantially outperforms general-purpose MLLMs in instruction quality and achieves state-of-the-art performance among open-source methods on reference image conditioned video editing. Our project webpage: https://rebind-mrv2v.github.io/.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 58 of its 62 references that are in the map. First posted on arXiv 16 Jul 2026.
Builds on
58 of 62 references.
Closest ancestors · 11
Same direction, newest first
- 2026TIDE: Task-Isolated Diffusion for Unified Video Editing and Generation6 Jun
- 2026Bernini: Latent Semantic Planning for Video Diffusion21 May
- 2026DreamVideo-Omni: Omni-Motion Controlled Multi-Subject Video Customization with Latent Identity Reinforcement Learning12 Mar
- 2025VEGGIE: Instructional Editing and Reasoning Video Concepts with Grounded Generation18 Mar
- 2025VACE: All-in-One Video Creation and Editing10 Mar
- 2025VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editing24 Feb
- 2025Improving Video Generation with Human Feedback23 Jan
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editing18 Aug
- 2023Dreamix: Video Diffusion Models are General Video Editors2 Feb
- 2022Video Diffusion Models7 Apr
Other references · 47
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,062
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2025SAM 3: Segment Anything with Conceptscited by 906
35 more
- 2025Qwen2.5-Omni Technical Reportcited by 736
- 2024LLaVA-Video: Video Instruction Tuning With Synthetic Datacited by 583
- 2025Qwen3-Omni Technical Reportcited by 497
- 2025MedGemma Technical Reportcited by 436
- 2024ShareGPT4Video: Improving Video Understanding and Generation with Better Captionscited by 388
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
- 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
- 2025Transfer between Modalities with MetaQueriescited by 236
- 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 236
- 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 140
- 2024video-SALMONN: Speech-Enhanced Audio-Visual Large Language Modelscited by 102
- 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 87
- 2025Kling-Omni Technical Reportcited by 71
- 2025InsViE-1M: Effective Instruction-based Video Editing with Elaborate Dataset Constructioncited by 62
- 2025Se~norita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialistscited by 61
- 2025UNIC: Unified In-Context Video Editingcited by 55
- 2025OpenVE-3M: A Large-Scale High-Quality Dataset for Instruction-Guided Video Editingcited by 53
- 2025video-SALMONN 2: Caption-Enhanced Audio-Visual Large Language Modelscited by 47
- 2023InstructVid2Vid: Controllable Video Editing with Natural Language Instructionscited by 44
- 2024VideoDirector: Precise Video Editing via Text-to-Video Modelscited by 36
- 2026Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidancecited by 36
- 2025Mavors: Multi-granularity Video Representation for Multimodal Large Language Modelcited by 32
- 2025VideoCap-R1: Enhancing MLLMs for Video Captioning via Structured Thinkingcited by 30
- 2025AVoCaDO: An Audiovisual Video Captioner Driven by Temporal Orchestrationcited by 27
- 2026OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoningcited by 27
- 2024StableV2V: Stablizing Shape Consistency in Video-to-Video Editingcited by 26
- 2026VINO: A Unified Visual Generator with Interleaved OmniModal Contextcited by 26
- 2025SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Modelscited by 22
- 2026DiaDem: Advancing Dialogue Descriptions in Audiovisual Video Captioning for Multimodal Large Language Modelscited by 13
- 2026UniVBench: Towards Unified Evaluation for Video Foundation Modelscited by 13
- 2026OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generationcited by 8
- 2025Towards Reason-Informed Video Editing in Unified Models with Self-Reflective Learningcited by 7
- 2026LoomVideo: Unifying Multimodal Inputs into Video Generation and Editingcited by 6
- 2025ViDiC: Video Difference Captioningcited by 2
- 2026Images in Sentences: Scaling Interleaved Instructions for Unified Visual Generationcited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026RefCaptioner: Multi-Reference Image-Grounded Video Captioningcited by 0
- 2026ContextMaster: Interactive Multi-Shot Video Creation via Fixed-Budget Sparse Context Routingcited by 0
- 2026PIGEONS: Incremental Long-Video Captioning with Persistent Identity Gallery and Evolving Overarching Narrative Summarycited by 0
- 2026PIGEONS: Incremental Long-Video Captioning with Persistent Identity Gallery and Evolving Overarching Narrative Summary12 Oct 2026
- 2026ContextMaster: Interactive Multi-Shot Video Creation via Fixed-Budget Sparse Context Routing5 Aug 2026
- 2026RefCaptioner: Multi-Reference Image-Grounded Video Captioning30 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2025InstructX: Towards Unified Visual Editing with MLLM Guidance
- 2026Tele-Omni: a Unified Multimodal Framework for Video Generation and Editing
- 2026MiVE: Multiscale Vision-language features for reference-guided video Editing
- 2026Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing
- 2026Omni-Video 2: Scaling MLLM-Conditioned Diffusion for Unified Video Generation and Editing
- 2026UniEditBench: A Unified and Cost-Effective Benchmark for Image and Video Editing via Distilled MLLMs
- 2025Are Image-to-Video Models Good Zero-Shot Image Editors?
- 2025VIVA: VLM-Guided Instruction-Based Video Editing with Reward Optimization