mlsanityData to 7 Oct 2026

Boximator: Generating Rich and Controllable Motions for Video Synthesis

arXiv 2402.01566 ↗First posted 2 Feb 2024cs.CV cs.AIcited by 107

Not on the map. Why?

Cited by 43 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generating rich and controllable motion is a pivotal challenge in video synthesis. We propose Boximator, a new approach for fine-grained motion control. Boximator introduces two constraint types: hard box and soft box. Users select objects in the conditional frame using hard boxes and then use either type of boxes to roughly or rigorously define the object's position, shape, or motion path in future frames. Boximator functions as a plug-in for existing video diffusion models. Its training process preserves the base model's knowledge by freezing the original weights and training only the control module. To address training challenges, we introduce a novel self-tracking technique that greatly simplifies the learning of box-object correlations. Empirically, Boximator achieves state-of-the-art video quality (FVD) scores, improving on two base models, and further enhanced after incorporating box constraints. Its robust motion controllability is validated by drastic increases in the bounding box alignment metric. Human evaluation also shows that users favor Boximator generation results over the base model.

Key facts. Cited by 107 papers in the map (data through 7 Oct 2026). Builds on 37 of its 45 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

37 of 45 references.

References · 37

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  6. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  7. 2022Video Diffusion Modelscited by 2,280
  8. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  9. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  10. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  11. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  12. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
25 more
  1. 2023ModelScope Text-to-Video Technical Reportcited by 651
  2. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  3. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 479
  4. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  5. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  6. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  7. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  8. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
  9. 2023Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generationcited by 300
  10. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  11. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
  12. 2021GODIVA: Generating Open-DomaIn Videos from nAtural Descriptionscited by 254
  13. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  14. 2023SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Predictioncited by 231
  15. 2023Tracking Anything with Decoupled Video Segmentationcited by 197
  16. 2022ReCo: Region-Controlled Text-to-Image Generationcited by 177
  17. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 172
  18. 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
  19. 2023TrailBlazer: Trajectory Control for Diffusion-Based Video Generationcited by 90
  20. 2018Grounded Video Descriptioncited by 78
  21. 2023Hierarchical Spatio-temporal Decoupling for Text-to-Video Generationcited by 50
  22. 2023Reuse and Diffuse: Iterative Denoising for Text-to-Video Generationcited by 49
  23. 2024MagicVideo-V2: Multi-Stage High-Aesthetic Video Generationcited by 48
  24. 2023DreaMoving: A Human Video Generation Framework based on Diffusion Modelscited by 23
  25. 2023Fine-grained Controllable Video Generation via Object Appearance and Contextcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 34; 2025: 41; 2026: 32.

107 papers.

Sort:

By area:

  1. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
  2. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
  3. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
  4. 2024IRASim: A Fine-Grained World Model for Robot Manipulationcited by 93
  5. 2025Controllable Video Generation: A Surveycited by 93
  6. 2024MotionBooth: Motion-Aware Customized Text-to-Video Generationcited by 92
  7. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 76
  8. 2024ReVideo: Remake a Video with Motion and Content Controlcited by 74
  9. 20243DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generationcited by 71
  10. 2024SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generationcited by 68
  11. 2025PhysGen3D: Crafting a Miniature Interactive World from a Single Imagecited by 65
  12. 2024Image Conductor: Precision Control for Interactive Video Synthesiscited by 62
38 more
  1. 2024Video Diffusion Models are Training-free Motion Interpreter and Controllercited by 60
  2. 2025MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidancecited by 60
  3. 2024FreeTraj: Tuning-Free Trajectory Control in Video Diffusion Modelscited by 55
  4. 2025MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generationcited by 50
  5. 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 49
  6. 2025Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidancecited by 45
  7. 2024DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Controlcited by 42
  8. 2025ATI: Any Trajectory Instruction for Controllable Video Generationcited by 42
  9. 2024TrackGo: A Flexible and Efficient Method for Controllable Video Generationcited by 39
  10. 2024Mind the Time: Temporally-Controlled Multi-Event Video Generationcited by 39
  11. 2025Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Controlcited by 33
  12. 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
  13. 2025ObjectMover: Generative Object Movement with Video Priorcited by 30
  14. 2025MotionPro: A Precise Motion Controller for Image-to-Video Generationcited by 29
  15. 2025Omni-Effects: Unified and Spatially-Controllable Visual Effects Generationcited by 27
  16. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Controlcited by 26
  17. 2025VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generationcited by 25
  18. 2024I2VControl: Disentangled and Unified Video Motion Synthesis Controlcited by 19
  19. 2026BridgeV2W: Bridging Video Generation Models to Embodied World Models via Embodiment Maskscited by 19
  20. 2024ObjCtrl-2.5D: Training-free Object Control with Camera Posescited by 18
  21. 2024MotionFlow: Attention-Driven Motion Transfer in Video Diffusion Modelscited by 16
  22. 2025BlobGEN-Vid: Compositional Text-to-Video Generation with Blob Video Representationscited by 15
  23. 2025Precise Action-to-Video Generation Through Visual Action Promptscited by 15
  24. 2025VideoHandles: Editing 3D Object Compositions in Videos Using Video Generative Priorscited by 14
  25. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applicationscited by 14
  26. 2024MotionShop: Zero-Shot Motion Transfer in Video Diffusion Models with Mixture of Score Guidancecited by 13
  27. 2025MotionV2V: Editing Motion in a Videocited by 13
  28. 2024EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolationcited by 12
  29. 20253D-Fixup: Advancing Photo Editing with 3D Priorscited by 12
  30. 2024Motion Modes: What Could Happen Next?cited by 11
  31. 2024Video Diffusion Transformers are In-Context Learnerscited by 11
  32. 2024InterDyn: Controllable Interactive Dynamics with Video Diffusion Modelscited by 11
  33. 2025AnyI2V: Animating Any Conditional Image with Motion Controlcited by 11
  34. 2024Animate Your Motion: Turning Still Images into Dynamic Videoscited by 10
  35. 2025AniCrafter: Customizing Realistic Human-Centric Animation via Avatar-Background Conditioning in Video Diffusion Modelscited by 10
  36. 2025Mask2IV: Interaction-Centric Video Generation via Mask Trajectoriescited by 10
  37. 2024ROICtrl: Boosting Instance Control for Visual Generationcited by 9
  38. 2025I2V3D: Controllable image-to-video generation with 3D guidancecited by 9

50 most cited of 107 All 107 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Ctrl-V: Higher Fidelity Video Generation with Bounding-Box Controlled Object Motion
  2. 2023Motion-Conditioned Diffusion Model for Controllable Video Synthesis
  3. 2026QWERTY: Training-Free Motion Control via Query-Warped Video Diffusion Transformers
  4. 2025Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoising
  5. 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motion
  6. 2024Boosting Camera Motion Control for Video Diffusion Transformers
  7. 2025LMP: Leveraging Motion Prior in Zero-Shot Video Generation with Diffusion Transformer
  8. 2024Controllable Longer Image Animation with Diffusion Models