mlsanityData to 14 Oct 2026

Get In Video: Add Anything You Want to the Video

arXiv 2503.06268 ↗First posted 8 Mar 2025cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Video editing increasingly demands the ability to incorporate specific real-world instances into existing footage, yet current approaches fundamentally fail to capture the unique visual characteristics of particular subjects and ensure natural instance/scene interactions. We formalize this overlooked yet critical editing paradigm as "Get-In-Video Editing", where users provide reference images to precisely specify visual elements they wish to incorporate into videos. Addressing this task's dual challenges, severe training data scarcity and technical challenges in maintaining spatiotemporal coherence, we introduce three key contributions. First, we develop GetIn-1M dataset created through our automated Recognize-Track-Erase pipeline, which sequentially performs video captioning, salient instance identification, object detection, temporal tracking, and instance removal to generate high-quality video editing pairs with comprehensive annotations (reference image, tracking mask, instance prompt). Second, we present GetInVideo, a novel end-to-end framework that leverages a diffusion transformer architecture with 3D full attention to process reference images, condition videos, and masks simultaneously, maintaining temporal coherence, preserving visual identity, and ensuring natural scene interactions when integrating reference objects into videos. Finally, we establish GetInBench, the first comprehensive benchmark for Get-In-Video Editing scenario, demonstrating our approach's superior performance through extensive evaluations. Our work enables accessible, high-quality incorporation of specific real-world subjects into videos, significantly advancing personalized video editing capabilities.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 32 of its 46 references that are in the map. First posted on arXiv 8 Mar 2025.

Builds on

32 of 46 references.

References · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2022Classifier-Free Diffusion Guidancecited by 5,847
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  7. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  9. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  10. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  11. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  12. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
20 more
  1. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  2. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 477
  3. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  4. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  5. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  6. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 314
  7. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 314
  8. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 256
  9. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 255
  10. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  11. 2023ProPainter: Improving Propagation and Transformer for Video Inpaintingcited by 197
  12. 2023CCEdit: Creative and Controllable Video Editing via Diffusion Modelscited by 82
  13. 2023FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesiscited by 79
  14. 2023MagicEdit: High-Fidelity and Temporally Coherent Video Editingcited by 76
  15. 2023Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Modelscited by 59
  16. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  17. 2025VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Controlcited by 42
  18. 2023DragVideo: Interactive Drag-style Video Editingcited by 40
  19. 2023MicroCinema: A Divide-and-Conquer Approach for Text-to-Video Generationcited by 26
  20. 2023Drag-A-Video: Non-rigid Video Editing with Point-based Interactioncited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 3.

12 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2024Temporally Consistent Object Editing in Videos using Extended Attention
  2. 2025FramePainter: Endowing Interactive Image Editing with Video Diffusion Priors
  3. 2023Edit-A-Video: Single Video Editing with Object-Aware Consistency
  4. 2026LIVE: Leveraging Image Manipulation Priors for Instruction-based Video Editing
  5. 2026VINCIE-NExT: Unlocking Video Editing from Images via In-Context Modeling
  6. 2025Zero-Shot Video Translation and Editing with Frame Spatial-Temporal Correspondence
  7. 2026OSVE: One Step Video Editing with One Step Diffusion Models
  8. 2024I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Models