mlsanityData to 7 Oct 2026

VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondence

arXiv 2312.02087 ↗First posted 4 Dec 2023cs.CVcited by 69

On the map: Diffusion models → Video generation

Abstract (arXiv)

Current diffusion-based video editing primarily focuses on structure-preserved editing by utilizing various dense correspondences to ensure temporal consistency and motion alignment. However, these approaches are often ineffective when the target edit involves a shape change. To embark on video editing with shape change, we explore customized video subject swapping in this work, where we aim to replace the main subject in a source video with a target subject having a distinct identity and potentially different shape. In contrast to previous methods that rely on dense correspondences, we introduce the VideoSwap framework that exploits semantic point correspondences, inspired by our observation that only a small number of semantic points are necessary to align the subject's motion trajectory and modify its shape. We also introduce various user-point interactions (eg, removing points and dragging points) to address various semantic point correspondence. Extensive experiments demonstrate state-of-the-art video subject swapping results across a variety of real-world videos.

Key facts. Cited by 69 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 60 of its 66 references that are in the map. First posted on arXiv 4 Dec 2023.

Builds on

60 of 66 references.

Closest ancestors · 13

Same direction, newest first

  1. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation27 Sep
  2. 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editing18 Aug
  3. 2023ModelScope Text-to-Video Technical Report12 Aug
  4. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning10 Jul
  5. 2023VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editing14 Jun
  6. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation13 Jun
  7. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models18 Apr
  8. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators23 Mar
  9. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editing16 Mar
  10. 2023Structure and Content-Guided Video Synthesis with Diffusion Models6 Feb
  11. 2023Dreamix: Video Diffusion Models are General Video Editors2 Feb
  12. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation22 Dec
1 more
  1. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct

Other references · 47

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2020Denoising Diffusion Implicit Modelscited by 9,655
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  8. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  9. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  10. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  11. 2023Consistency Modelscited by 1,778
  12. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
35 more
  1. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  2. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  3. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  4. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  5. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  6. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  7. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  8. 2023CoTracker: It is Better to Track Togethercited by 516
  9. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  10. 2023Emergent Correspondence from Image Diffusioncited by 437
  11. 2022Blended Latent Diffusioncited by 430
  12. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  13. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  14. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  15. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  16. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 293
  17. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 281
  18. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  19. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  20. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  21. 2023HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Modelscited by 224
  22. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 218
  23. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 193
  24. 2023Tracking Everything Everywhere All at Oncecited by 190
  25. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  26. 2023Taming Encoder for Zero Fine-tuning Image Customization with Text-to-Image Diffusion Modelscited by 116
  27. 2023CoDeF: Content Deformation Fields for Temporally Consistent Video Processingcited by 108
  28. 2023MagicEdit: High-Fidelity and Temporally Coherent Video Editingcited by 75
  29. 2023SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Modelscited by 71
  30. 2023Shape-aware Text-driven Layered Video Editingcited by 54
  31. 2023Custom-Edit: Text-Guided Image Editing with Customized Diffusion Modelscited by 54
  32. 2023CVPR 2023 Text Guided Video Editing Competitioncited by 54
  33. 2023Photoswap: Personalized Subject Swapping in Imagescited by 47
  34. 2023DreamEdit: Subject-driven Image Editingcited by 37
  35. 2023INVE: Interactive Neural Video Editingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 30; 2026: 15.

69 papers.

Sort:

By area:

  1. 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motioncited by 167
  2. 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
  3. 2024AnyV2V: A Tuning-Free Framework For Any Video-to-Video Editing Taskscited by 151
  4. 2025InsViE-1M: Effective Instruction-based Video Editing with Elaborate Dataset Constructioncited by 62
  5. 2024COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editingcited by 59
  6. 2024I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Modelscited by 55
  7. 2024Towards A Better Metric for Text-to-Video Generationcited by 50
  8. 2025VideoGrain: Modulating Space-Time Attention for Multi-grained Video Editingcited by 50
  9. 2024UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editingcited by 47
  10. 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
  11. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  12. 2024Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditionscited by 42
38 more
  1. 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generationcited by 42
  2. 2025VideoAnydoor: High-fidelity Video Object Insertion with Precise Motion Controlcited by 42
  3. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 40
  4. 2024Generative Video Propagationcited by 36
  5. 2024DisPose: Disentangling Pose Guidance for Controllable Human Image Animationcited by 35
  6. 2025ContextFlow: Training-Free Video Object Editing via Adaptive Context Enrichmentcited by 27
  7. 2025FlowDirector: Training-Free Flow Steering for Precise Text-to-Video Editingcited by 24
  8. 2024HOI-Swap: Swapping Objects in Videos with Hand-Object Interaction Awarenesscited by 21
  9. 2025Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Modelcited by 19
  10. 2024TAPTRv2: Attention-based Position Update Improves Tracking Any Pointcited by 18
  11. 2024EVA: Zero-shot Accurate Attributes and Multi-Object Video Editingcited by 17
  12. 2024DIVE: Taming DINO for Subject-Driven Video Editingcited by 17
  13. 2024SwapAnything: Enabling Arbitrary Object Swapping in Personalized Visual Editingcited by 13
  14. 2024DeCo: Decoupled Human-Centered Diffusion Video Editing with Motion Consistencycited by 13
  15. 2025Fast Multi-view Consistent 3D Editing with Video Priorscited by 12
  16. 2024CTRL-D: Controllable Dynamic 3D Scene Editing with Personalized 2D Diffusioncited by 11
  17. 2024InVi: Object Insertion In Videos Using Off-the-Shelf Diffusion Modelscited by 10
  18. 2025Direct Motion Models for Assessing Generated Videoscited by 10
  19. 2024DragEntity: Trajectory Guided Video Generation using Entity and Positional Relationshipscited by 9
  20. 2024ROICtrl: Boosting Instance Control for Visual Generationcited by 9
  21. 2024EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Modelscited by 8
  22. 2025Visual Prompting for One-shot Controllable Video Editing without Inversioncited by 8
  23. 2025Shape-for-Motion: Precise and Consistent Video Editing with 3D Proxycited by 8
  24. 2025DreamInsert: Zero-Shot Image-to-Video Object Insertion from A Single Imagecited by 6
  25. 2025AdaFlow: Efficient Long Video Editing via Adaptive Attention Slimming And Keyframe Selectioncited by 5
  26. 2025JointTuner: Appearance-Motion Adaptive Joint Training for Customized Video Generationcited by 5
  27. 2025Causally Steered Diffusion for Automated Video Counterfactual Generationcited by 5
  28. 2025VideoCoF: Unified Video Editing with Temporal Reasonercited by 5
  29. 2026FFP-300K: Scaling First-Frame Propagation for Generalizable Video Editingcited by 5
  30. 2025IC-Effect: Precise and Efficient Video Effects Editing via In-Context Learningcited by 4
  31. 2025ProTracker: Probabilistic Integration for Robust and Accurate Point Trackingcited by 3
  32. 2025iDiT-HOI: Inpainting-based Hand Object Interaction Reenactment via Video Diffusion Transformercited by 3
  33. 2025DFVEdit: Conditional Delta Flow Vector for Zero-shot Video Editingcited by 3
  34. 2026SteerVTE: Seamless Video Text Editing with Style and Glyph Controlcited by 3
  35. 2025Weakly-Supervised Learning of Dense Functional Correspondencescited by 2
  36. 2025Bring Your Dreams to Life: Continual Text-to-Video Customizationcited by 2
  37. 2026Memory-V2V: Memory-Augmented Video-to-Video Diffusion for Consistent Multi-Turn Editingcited by 2
  38. 2026TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Maskingcited by 2

50 most cited of 69 All 69 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023MotionEditor: Editing Video Motion via Content-Aware Diffusion
  2. 2025VideoMage: Multi-Subject and Motion Customization of Text-to-Video Diffusion Models
  3. 2026MotionAdapter: Video Motion Transfer via Content-Aware Attention Customization
  4. 2025FramePainter: Endowing Interactive Image Editing with Video Diffusion Priors
  5. 2025Zero-Shot Video Translation and Editing with Frame Spatial-Temporal Correspondence
  6. 2024Temporally Consistent Object Editing in Videos using Extended Attention
  7. 2026Beyond Consistency: Preserving Temporal Structure in Zero-Shot Video Editing
  8. 2024Animate Your Motion: Turning Still Images into Dynamic Videos