mlsanityData to 14 Oct 2026

DragAnything: Motion Control for Anything using Entity Representation

arXiv 2403.07420 ↗First posted 12 Mar 2024cs.CVcited by 159

Not on the map. Why?

Cited by 51 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce DragAnything, which utilizes a entity representation to achieve motion control for any object in controllable video generation. Comparison to existing motion control methods, DragAnything offers several advantages. Firstly, trajectory-based is more userfriendly for interaction, when acquiring other guidance signals (e.g., masks, depth maps) is labor-intensive. Users only need to draw a line (trajectory) during interaction. Secondly, our entity representation serves as an open-domain embedding capable of representing any object, enabling the control of motion for diverse entities, including background. Lastly, our entity representation allows simultaneous and distinct motion control for multiple objects. Extensive experiments demonstrate that our DragAnything achieves state-of-the-art performance for FVD, FID, and User Study, particularly in terms of object motion control, where our method surpasses the previous methods (e.g., DragNUWA) by 26% in human voting.

Key facts. Cited by 159 papers in the map (data through 14 Oct 2026). Builds on 49 of its 54 references that are in the map. First posted on arXiv 12 Mar 2024.

Builds on

49 of 54 references.

References · 49

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2023Segment Anythingcited by 8,234
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  10. 2022Video Diffusion Modelscited by 2,305
  11. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  12. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
37 more
  1. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  2. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  3. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  4. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  5. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  6. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  7. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 535
  8. 2023CoTracker: It is Better to Track Togethercited by 522
  9. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 465
  10. 2023Emergent Correspondence from Image Diffusioncited by 446
  11. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 412
  12. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 350
  13. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  14. 2023ControlVideo: Training-free Controllable Text-to-Video Generationcited by 327
  15. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 314
  16. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 302
  17. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 279
  18. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 264
  19. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  20. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 255
  21. 2023A Survey on Video Diffusion Modelscited by 206
  22. 2023SegGPT: Segmenting Everything In Contextcited by 194
  23. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 175
  24. 2023MotionDirector: Motion Customization of Text-to-Video Diffusion Modelscited by 173
  25. 2023PTQD: Accurate Post-Training Quantization for Diffusion Modelscited by 170
  26. 2023DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Modelscited by 162
  27. 2023RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Pathscited by 142
  28. 2023DatasetDM: Synthesizing Data with Perception Annotations Using Diffusion Modelscited by 123
  29. 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesiscited by 99
  30. 2023TrailBlazer: Trajectory Control for Diffusion-Based Video Generationcited by 91
  31. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Modelscited by 78
  32. 2023VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondencecited by 69
  33. 2021iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesiscited by 42
  34. 2024Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditionscited by 42
  35. 2023Paragraph-to-Image Generation with Information-Enriched Diffusion Modelcited by 39
  36. 2021Understanding Object Dynamics for Interactive Image-to-Video Synthesiscited by 38
  37. 2021Click to Move: Controlling Video Generation with Sparse Motioncited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 27; 2025: 76; 2026: 56.

159 papers.

Sort:

By area:

  1. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 178
  2. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 164
  3. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 154
  4. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 117
  5. 2025Automated Movie Generation via Multi-Agent CoT Planningcited by 89
  6. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 82
  7. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 76
  8. 2024ReVideo: Remake a Video with Motion and Content Controlcited by 74
  9. 2024SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generationcited by 68
  10. 2025PhysGen3D: Crafting a Miniature Interactive World from a Single Imagecited by 66
  11. 2024Image Conductor: Precision Control for Interactive Video Synthesiscited by 63
  12. 2025MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidancecited by 61
38 more
  1. 2025Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signalscited by 53
  2. 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 52
  3. 2025MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generationcited by 50
  4. 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
  5. 2025WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actionscited by 45
  6. 2025Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidancecited by 45
  7. 2025ATI: Any Trajectory Instruction for Controllable Video Generationcited by 43
  8. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 42
  9. 2025PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generationcited by 41
  10. 2024TrackGo: A Flexible and Efficient Method for Controllable Video Generationcited by 39
  11. 2024LeviTor: 3D Trajectory Oriented Image-to-Video Synthesiscited by 39
  12. 2025TAPNext: Tracking Any Point (TAP) as Next Token Predictioncited by 37
  13. 2024MovieBench: A Hierarchical Movie Level Dataset for Long Video Generationcited by 36
  14. 2024AnimateAnything: Consistent and Controllable Animation for Video Generationcited by 35
  15. 2024DisPose: Disentangling Pose Guidance for Controllable Human Image Animationcited by 35
  16. 2024Generative Image Layer Decomposition with Visual Effectscited by 34
  17. 2025Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Controlcited by 34
  18. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  19. 2025ObjectMover: Generative Object Movement with Video Priorcited by 30
  20. 2025MotionPro: A Precise Motion Controller for Image-to-Video Generationcited by 30
  21. 2024Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Modelcited by 28
  22. 2024Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamicscited by 28
  23. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Controlcited by 28
  24. 2025VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generationcited by 25
  25. 2024Framer: Interactive Frame Interpolationcited by 24
  26. 2025Perception-as-Control: Fine-grained Controllable Image Animation with 3D-aware Motion Representationcited by 23
  27. 2025Generative Video Motion Editing with 3D Point Trackscited by 23
  28. 2024AniDoc: Animation Creation Made Easiercited by 22
  29. 2026RealWonder: Real-Time Physical Action-Conditioned Video Generationcited by 22
  30. 2025EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidancecited by 21
  31. 2024ObjCtrl-2.5D: Training-free Object Control with Camera Posescited by 18
  32. 2025X-Dyna: Expressive Dynamic Human Image Animationcited by 18
  33. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  34. 2025MotionV2V: Editing Motion in a Videocited by 14
  35. 2025BulletTime: Decoupled Control of Time and Camera Pose for Video Generationcited by 14
  36. 2026Physical Simulator In-the-Loop Video Generationcited by 14
  37. 2025Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoisingcited by 13
  38. 2024EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolationcited by 12

50 most cited of 159 All 159 (JSON)