# InstantDrag: Improving Interactivity in Drag-based Image Editing

arXiv: https://arxiv.org/abs/2409.08857 · HTML: https://mlsanity.com/paper/2409.08857/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 52 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [Pivotal Tuning for Latent-based Editing of Real Images](https://mlsanity.com/paper/2106.05744/) (2021)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold](https://mlsanity.com/paper/2305.10973/) (2023)
- [LCM-LoRA: A Universal Stable-Diffusion Acceleration Module](https://mlsanity.com/paper/2311.05556/) (2023)
- [CelebV-Text: A Large-Scale Facial Text-Video Dataset](https://mlsanity.com/paper/2303.14717/) (2023)
- [Distilling Diffusion Models into Conditional GANs](https://mlsanity.com/paper/2405.05967/) (2024)
- [Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D](https://mlsanity.com/paper/2312.02190/) (2023)
- [Drag Your Noise: Interactive Point-based Editing via Diffusion Semantic Propagation](https://mlsanity.com/paper/2404.01050/) (2024)
- [Motion Guidance: Diffusion-Based Image Editing with Differentiable Motion Estimators](https://mlsanity.com/paper/2401.18085/) (2024)
- [Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videos](https://mlsanity.com/paper/2403.13044/) (2024)
- [GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models](https://mlsanity.com/paper/2404.07206/) (2024)
- [DragAPart: Learning a Part-Level Motion Prior for Articulated Objects](https://mlsanity.com/paper/2403.15382/) (2024)
- [StableDrag: Stable Dragging for Point-based Image Editing](https://mlsanity.com/paper/2403.04437/) (2024)
- [Self-Supervised Learning via Conditional Motion Propagation](https://mlsanity.com/paper/1903.11412/) (2019)

## Built on by (most cited first, 30 of 31)

- [MotionStream: Real-Time Video Generation with Interactive Motion Controls](https://mlsanity.com/paper/2511.01266/) (2025)
- [DragFlow: Unleashing DiT Priors with Region Based Supervision for Drag Editing](https://mlsanity.com/paper/2510.02253/) (2025)
- [VINCIE: Unlocking In-context Image Editing from Video](https://mlsanity.com/paper/2506.10941/) (2025)
- [VideoHandles: Editing 3D Object Compositions in Videos Using Video Generative Priors](https://mlsanity.com/paper/2503.01107/) (2025)
- [CLIPDrag: Combining Text-based and Drag-based Instructions for Image Editing](https://mlsanity.com/paper/2410.03097/) (2024)
- [MvDrag3D: Drag-based Creative 3D Editing via Multi-view Generation-Reconstruction Priors](https://mlsanity.com/paper/2410.16272/) (2024)
- [Training-free Geometric Image Editing on Diffusion Models](https://mlsanity.com/paper/2507.23300/) (2025)
- [3D-Fixup: Advancing Photo Editing with 3D Priors](https://mlsanity.com/paper/2505.10566/) (2025)
- [DragLoRA: Online Optimization of LoRA Adapters for Drag-based Image Editing in Diffusion Model](https://mlsanity.com/paper/2505.12427/) (2025)
- [AdaptiveDrag: Semantic-Driven Dragging on Diffusion-Based Image Editing](https://mlsanity.com/paper/2410.12696/) (2024)
- [MagicFace: High-Fidelity Facial Expression Editing with Action-Unit Control](https://mlsanity.com/paper/2501.02260/) (2025)
- [Target-Aware Video Diffusion Models](https://mlsanity.com/paper/2503.18950/) (2025)
- [Inpaint4Drag: Repurposing Inpainting Models for Drag-Based Image Editing via Bidirectional Warping](https://mlsanity.com/paper/2509.04582/) (2025)
- [Dragging with Geometry: From Pixels to Geometry-Guided Image Editing](https://mlsanity.com/paper/2509.25740/) (2025)
- [Neural-Driven Image Editing](https://mlsanity.com/paper/2507.05397/) (2025)
- [LazyDrag: Enabling Stable Drag-Based Editing on Multi-Modal Diffusion Transformers via Explicit Correspondence](https://mlsanity.com/paper/2509.12203/) (2025)
- [BlobCtrl: Taming Controllable Blob for Element-level Image Editing](https://mlsanity.com/paper/2503.13434/) (2025)
- [What If : Understanding Motion Through Sparse Interactions](https://mlsanity.com/paper/2510.12777/) (2025)
- [Talk2Move: Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenes](https://mlsanity.com/paper/2601.02356/) (2026)
- [Envisioning the Future, One Step at a Time](https://mlsanity.com/paper/2604.09527/) (2026)
- [DirectDrag: High-Fidelity, Mask-Free, Prompt-Free Drag-based Image Editing via Readout-Guided Feature Alignment](https://mlsanity.com/paper/2512.03981/) (2025)
- [ContextDrag: Precise Drag-Based Image Editing via Context-Preserving Token Injection and Position-Aligned Attention](https://mlsanity.com/paper/2512.08477/) (2025)
- [RealDrag: The First Dragging Benchmark with Real Target Image](https://mlsanity.com/paper/2512.12287/) (2025)
- [FG-Portrait: 3D Flow Guided Editable Portrait Animation](https://mlsanity.com/paper/2603.23381/) (2026)
- [Learning Long-term Motion Embeddings for Efficient Kinematics Generation](https://mlsanity.com/paper/2604.11737/) (2026)
- [DynaDrag: Dynamic Drag-Style Image Editing by Motion Prediction](https://mlsanity.com/paper/2601.00542/) (2026)
- [TRACE: Structure-Aware Character Encoding for Robust and Generalizable Document Watermarking](https://mlsanity.com/paper/2603.12873/) (2026)
- [Editing Away the Evidence: Diffusion-Based Image Manipulation and the Failure Modes of Robust Watermarking](https://mlsanity.com/paper/2603.12949/) (2026)
- [Drag within Prior Distribution: Text-Conditioned Point-Based Image Editing within Distribution Constraints](https://mlsanity.com/paper/2605.13349/) (2026)
- [In-context Region-based Drag: Drag Any Region to Any Shape](https://mlsanity.com/paper/2606.25907/) (2026)

## Abstract (arXiv)

Drag-based image editing has recently gained popularity for its interactivity and precision. However, despite the ability of text-to-image models to generate samples within a second, drag editing still lags behind due to the challenge of accurately reflecting user interaction while maintaining image content. Some existing approaches rely on computationally intensive per-image optimization or intricate guidance-based methods, requiring additional inputs such as masks for movable regions and text prompts, thereby compromising the interactivity of the editing process. We introduce InstantDrag, an optimization-free pipeline that enhances interactivity and speed, requiring only an image and a drag instruction as input. InstantDrag consists of two carefully designed networks: a drag-conditioned optical flow generator (FlowGen) and an optical flow-conditioned diffusion model (FlowDiffusion). InstantDrag learns motion dynamics for drag-based image editing in real-world video datasets by decomposing the task into motion generation and motion-conditioned image generation. We demonstrate InstantDrag's capability to perform fast, photo-realistic edits without masks or text prompts through experiments on facial video datasets and general scenes. These results highlight the efficiency of our approach in handling drag-based image editing, making it a promising solution for interactive, real-time applications.