mlsanityData to 14 Oct 2026

Shape-aware Text-driven Layered Video Editing

arXiv 2301.13173 ↗First posted 30 Jan 2023cs.CV eess.IVcited by 54

Not on the map. Why?

Cited by 28 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Temporal consistency is essential for video editing applications. Existing work on layered representation of videos allows propagating edits consistently to each frame. These methods, however, can only edit object appearance rather than object shape changes due to the limitation of using a fixed UV mapping field for texture atlas. We present a shape-aware, text-driven video editing method to tackle this challenge. To handle shape changes in video editing, we first propagate the deformation field between the input and edited keyframe to all frames. We then leverage a pre-trained text-conditioned diffusion model as guidance for refining shape distortion and completing unseen regions. The experimental results demonstrate that our method can achieve shape-aware consistent video editing and compare favorably with the state-of-the-art.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Builds on 55 of its 64 references that are in the map. First posted on arXiv 30 Jan 2023.

Builds on

55 of 64 references.

References · 55

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  11. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  12. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
38 more
  1. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  2. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  3. 2015Spatial Transformer Networkscited by 1,429
  4. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  5. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  6. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
  7. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 957
  8. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  9. 2016Generative Adversarial Text to Image Synthesiscited by 735
  10. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  11. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 642
  12. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  13. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  14. 2022Text2LIVE: Text-Driven Layered Image and Video Editingcited by 312
  15. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 301
  16. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  17. 2018Learning Blind Video Temporal Consistencycited by 253
  18. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  19. 2020Swapping Autoencoder for Deep Image Manipulationcited by 215
  20. 2021CLIPstyler: Image Style Transfer with a Single Text Conditioncited by 203
  21. 2021CLIPDraw: Exploring Text-to-Drawing Synthesis through Language-Image Encoderscited by 187
  22. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  23. 2019Style Transfer by Relaxed Optimal Transport and Self-Similaritycited by 168
  24. 2021StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generatorscited by 164
  25. 2019ManiGAN: Text-Guided Image Manipulationcited by 156
  26. 2022FILM: Frame Interpolation for Large Motioncited by 153
  27. 2021Layered Neural Atlases for Consistent Video Editingcited by 137
  28. 2019Object-driven Text-to-Image Synthesis via Adversarial Trainingcited by 117
  29. 2022Towards An End-to-End Framework for Flow-Guided Video Inpaintingcited by 113
  30. 2022Generating Long Videos of Dynamic Scenescited by 113
  31. 2020Flow-edge Guided Video Completioncited by 104
  32. 2018Text-Adaptive Generative Adversarial Networks: Manipulating Images with Natural Languagecited by 90
  33. 2021FuseDream: Training-Free Text-to-Image Generation with Improved CLIP+GAN Space Optimizationcited by 71
  34. 2022Deformable Sprites for Unsupervised Video Decompositioncited by 61
  35. 2021Text to Image Generation with Semantic-Spatial Aware GANcited by 53
  36. 2020Layered Neural Rendering for Retiming People in Videocited by 51
  37. 2021Omnimatte: Associating Objects and Their Effects in Videocited by 50
  38. 2021Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Modelcited by 34

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 22; 2024: 17; 2025: 11; 2026: 4.

54 papers.

Sort:

By area:

  1. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  2. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 670
  3. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 466
  4. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 442
  5. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
  6. 2023A Survey on Video Diffusion Modelscited by 206
  7. 2024Taming Rectified Flow for Inversion and Editingcited by 204
  8. 2023State of the Art on Diffusion Models for Visual Computingcited by 127
  9. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
  10. 2023RAVE: Randomized Noise Shuffling for Fast and Consistent Video Editing with Diffusion Modelscited by 114
  11. 2023FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesiscited by 79
  12. 2023VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondencecited by 69
38 more
  1. 2023Edit-A-Video: Single Video Editing with Object-Aware Consistencycited by 67
  2. 2023MagicStick: Controllable Video Editing via Control Handle Transformationscited by 62
  3. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  4. 2023Fairy: Fast Parallelized Instruction-Guided Video-to-Video Synthesiscited by 39
  5. 2024Generative Omnimatte: Learning to Decompose Video into Layerscited by 38
  6. 2023VidEdit: Zero-Shot and Spatially Aware Text-Driven Video Editingcited by 36
  7. 2024Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Videocited by 26
  8. 2024StableV2V: Stablizing Shape Consistency in Video-to-Video Editingcited by 26
  9. 2025Generative Video Motion Editing with 3D Point Trackscited by 23
  10. 2023Hulk: A Universal Knowledge Translator for Human-Centric Taskscited by 22
  11. 2024HOI-Swap: Swapping Objects in Videos with Hand-Object Interaction Awarenesscited by 21
  12. 2024VASE: Object-Centric Appearance and Shape Manipulation of Real Videoscited by 20
  13. 2025Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Modelcited by 19
  14. 2023DynVideo-E: Harnessing Dynamic NeRF for Large-Scale Motion- and View-Change Human-Centric Video Editingcited by 17
  15. 2023InstructP2P: Learning to Edit 3D Point Clouds with Text Instructionscited by 14
  16. 2023OmnimatteRF: Robust Omnimatte with 3D Background Modelingcited by 14
  17. 2025Zero-Shot Audio-Visual Editing via Cross-Modal Delta Denoisingcited by 14
  18. 2024DeCo: Decoupled Human-Centered Diffusion Video Editing with Motion Consistencycited by 13
  19. 2024Edit-Your-Motion: Space-Time Diffusion Decoupling Learning for Video Motion Editingcited by 10
  20. 2025Synchronized Video-to-Audio Generation via Mel Quantization-Continuum Decompositioncited by 10
  21. 2024Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunitiescited by 8
  22. 2025V2Edit: Versatile Video Diffusion Editor for Videos and 3D Scenescited by 8
  23. 2025Visual Prompting for One-shot Controllable Video Editing without Inversioncited by 8
  24. 2024GenVideo: One-shot Target-image and Shape Aware Video Editing using T2I Diffusion Modelscited by 7
  25. 2026Evolution of Video Generative Foundationscited by 7
  26. 2023DiffusionAtlas: High-Fidelity Consistent Diffusion Video Editingcited by 6
  27. 2023EVE: Efficient zero-shot text-based Video Editing with Depth Map Guidance and Temporal Consistency Constraintscited by 5
  28. 2025iDiT-HOI: Inpainting-based Hand Object Interaction Reenactment via Video Diffusion Transformercited by 5
  29. 2024AnimateZoo: Zero-shot Video Generation of Cross-Species Animation via Subject Alignmentcited by 3
  30. 2024Shaping a Stabilized Video by Mitigating Unintended Changes for Concept-Augmented Video Editingcited by 3
  31. 2023RealCraft: Attention Control as A Tool for Zero-Shot Consistent Video Editingcited by 2
  32. 2024Ada-VE: Training-Free Consistent Video Editing Using Adaptive Motion Priorcited by 2
  33. 2024Fast Sprite Decomposition from Animated Graphicscited by 2
  34. 2024Beyond Generation: Unlocking Universal Editing via Self-Supervised Fine-Tuningcited by 2
  35. 2024Blended Latent Diffusion under Attention Control for Real-World Video Editingcited by 1
  36. 2025SST-EM: Advanced Metrics for Evaluating Semantic, Spatial and Temporal Aspects in Video Editingcited by 1
  37. 2025HALO: Human-Aligned End-to-end Image Retargeting with Layered Transformationscited by 1
  38. 2025Edit-Your-Interest: Efficient Video Editing via Feature Most-Similar Propagationcited by 1

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editing
  2. 2025Shape-for-Motion: Precise and Consistent Video Editing with 3D Proxy
  3. 2023ControlVideo: Conditional Control for One-shot Text-driven Video Editing and Beyond
  4. 2023Structure and Content-Guided Video Synthesis with Diffusion Models
  5. 2023Edit Temporal-Consistent Videos with Image Diffusion Model
  6. 2025Consistent Video Editing as Flow-Driven Image-to-Video Generation
  7. 2026Vera: A Layered Diffusion Model for Content-Preserving Video Editing
  8. 2026Edit-VAR: Taming Visual Autoregressive Model for Precise Video Editing