# LightningDrag: Lightning Fast and Accurate Drag-based Image Editing Emerging from Videos

arXiv: https://arxiv.org/abs/2405.13722 · HTML: https://mlsanity.com/paper/2405.13722/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 70 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 30 of 60)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing](https://mlsanity.com/paper/2304.08465/) (2023)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [Zero-shot Image-to-Image Translation](https://mlsanity.com/paper/2302.03027/) (2023)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [AnyDoor: Zero-shot Object-level Image Customization](https://mlsanity.com/paper/2307.09481/) (2023)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)

## Built on by (most cited first, 19 of 19)

- [DragFlow: Unleashing DiT Priors with Region Based Supervision for Drag Editing](https://mlsanity.com/paper/2510.02253/) (2025)
- [FramePainter: Endowing Interactive Image Editing with Video Diffusion Priors](https://mlsanity.com/paper/2501.08225/) (2025)
- [Motion Modes: What Could Happen Next?](https://mlsanity.com/paper/2412.00148/) (2024)
- [DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model](https://mlsanity.com/paper/2505.12427/) (2025)
- [AdaptiveDrag: Semantic-Driven Dragging on Diffusion-Based Image Editing](https://mlsanity.com/paper/2410.12696/) (2024)
- [3D Scene Understanding Through Local Random Access Sequence Modeling](https://mlsanity.com/paper/2504.03875/) (2025)
- [LazyDrag: Enabling Stable Drag-Based Editing on Multi-Modal Diffusion Transformers via Explicit Correspondence](https://mlsanity.com/paper/2509.12203/) (2025)
- [CraftMesh: High-Fidelity Generative Mesh Manipulation via Poisson Seamless Fusion](https://mlsanity.com/paper/2509.13688/) (2025)
- [Discovering and using Spelke segments](https://mlsanity.com/paper/2507.16038/) (2025)
- [Unified 3D Scene Understanding Through Physical World Modeling](https://mlsanity.com/paper/2605.24321/) (2026)
- [DirectDrag: High-Fidelity, Mask-Free, Prompt-Free Drag-based Image Editing via Readout-Guided Feature Alignment](https://mlsanity.com/paper/2512.03981/) (2025)
- [RealDrag: The First Dragging Benchmark with Real Target Image](https://mlsanity.com/paper/2512.12287/) (2025)
- [Geometric Image Editing via Effects-Sensitive In-Context Inpainting with Diffusion Transformers](https://mlsanity.com/paper/2602.08388/) (2026)
- [Text-Vision Co-Instructed Image Editing](https://mlsanity.com/paper/2606.16767/) (2026)
- [In-context Region-based Drag: Drag Any Region to Any Shape](https://mlsanity.com/paper/2606.25907/) (2026)
- [Perceptual 3D Simulation With Physical World Modeling](https://mlsanity.com/paper/2606.27575/) (2026)
- [G-Skin: Learning to Bind 3D Gaussians with Generative Visual Priors](https://mlsanity.com/paper/2608.01726/) (2026)
- [Drag as Evidence: Motion-Grounded Latent Recomposition for Drag-Based Editing](https://mlsanity.com/paper/2609.36755/) (2026)
- [ThinkDrag: Semantic Drag-Based Image Editing with Visual Reasoning](https://mlsanity.com/paper/2610.13316/) (2026)

## Abstract (arXiv)

Accuracy and speed are critical in image editing tasks. Pan et al. introduced a drag-based image editing framework that achieves pixel-level control using Generative Adversarial Networks (GANs). A flurry of subsequent studies enhanced this framework's generality by leveraging large-scale diffusion models. However, these methods often suffer from inordinately long processing times (exceeding 1 minute per edit) and low success rates. Addressing these issues head on, we present LightningDrag, a rapid approach enabling high quality drag-based image editing in ~1 second. Unlike most previous methods, we redefine drag-based editing as a conditional generation task, eliminating the need for time-consuming latent optimization or gradient-based guidance during inference. In addition, the design of our pipeline allows us to train our model on large-scale paired video frames, which contain rich motion information such as object translations, changing poses and orientations, zooming in and out, etc. By learning from videos, our approach can significantly outperform previous methods in terms of accuracy and consistency. Despite being trained solely on videos, our model generalizes well to perform local shape deformations not presented in the training data (e.g., lengthening of hair, twisting rainbows, etc.). Extensive qualitative and quantitative evaluations on benchmark datasets corroborate the superiority of our approach. The code and model will be released at https://github.com/magic-research/LightningDrag.