mlsanityData to 7 Oct 2026

Image Conductor: Precision Control for Interactive Video Synthesis

arXiv 2406.15339 ↗First posted 21 Jun 2024cs.CV cs.AI cs.MMcited by 62

Not on the map. Why?

Abstract (arXiv)

Filmmaking and animation production often require sophisticated techniques for coordinating camera transitions and object movements, typically involving labor-intensive real-world capturing. Despite advancements in generative AI for video creation, achieving precise control over motion for interactive video asset generation remains challenging. To this end, we propose Image Conductor, a method for precise control of camera transitions and object movements to generate video assets from a single image. An well-cultivated training strategy is proposed to separate distinct camera and object motion by camera LoRA weights and object LoRA weights. To further address cinematographic variations from ill-posed trajectories, we introduce a camera-free guidance technique during inference, enhancing object movements while eliminating camera transitions. Additionally, we develop a trajectory-oriented video motion data curation pipeline for training. Quantitative and qualitative experiments demonstrate our method's precision and fine-grained control in generating motion-controllable videos from images, advancing the practical application of interactive video synthesis. Project webpage available at https://liyaowei-stu.github.io/project/ImageConductor/

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Builds on 38 of its 43 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

38 of 43 references.

References · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  10. 2022Classifier-Free Diffusion Guidancecited by 5,717
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
26 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  4. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  6. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  7. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  8. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  9. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  10. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
  11. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generationcited by 530
  12. 2023CoTracker: It is Better to Track Togethercited by 516
  13. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  14. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  15. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
  16. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
  17. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
  18. 2021GMFlow: Learning Optical Flow via Global Matchingcited by 324
  19. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
  20. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  21. 2021Conditional Image Generation with Score-Based Diffusion Modelscited by 195
  22. 2023SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Modelscited by 172
  23. 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
  24. 2024Boximator: Generating Rich and Controllable Motions for Video Synthesiscited by 107
  25. 2022ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wildcited by 90
  26. 2024Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuningcited by 59

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 32; 2026: 20.

62 papers.

Sort:

By area:

  1. 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
  2. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
  3. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 114
  4. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  5. 2024SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generationcited by 68
  6. 2025MagicMotion: Controllable Video Generation with Dense-to-Sparse Trajectory Guidancecited by 60
  7. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 54
  8. 2025Force Prompting: Video Generation Models Can Learn and Generalize Physics-based Control Signalscited by 51
  9. 2025Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidancecited by 45
  10. 2025WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actionscited by 43
  11. 2024DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Controlcited by 42
  12. 2025ATI: Any Trajectory Instruction for Controllable Video Generationcited by 42
38 more
  1. 2024DisPose: Disentangling Pose Guidance for Controllable Human Image Animationcited by 35
  2. 2025TAPNext: Tracking Any Point (TAP) as Next Token Predictioncited by 35
  3. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  4. 2024Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamicscited by 28
  5. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Controlcited by 26
  6. 2025VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generationcited by 25
  7. 2025Perception-as-Control: Fine-grained Controllable Image Animation with 3D-aware Motion Representationcited by 22
  8. 2026BridgeV2W: Bridging Video Generation Models to Embodied World Models via Embodiment Maskscited by 19
  9. 2024ObjCtrl-2.5D: Training-free Object Control with Camera Posescited by 18
  10. 2025Free-Form Motion Control: Controlling the 6D Poses of Camera and Objects in Video Generationcited by 15
  11. 2025SketchVideo: Sketch-based Video Generation and Editingcited by 15
  12. 2025Simulating the Visual World with Artificial Intelligence: A Roadmapcited by 15
  13. 2025Time-to-Move: Training-Free Motion Controlled Video Generation via Dual-Clock Denoisingcited by 13
  14. 2025MotionV2V: Editing Motion in a Videocited by 13
  15. 2026Physical Simulator In-the-Loop Video Generationcited by 12
  16. 2025MotionAgent: Fine-grained Controllable Video Generation via Motion Field Agentcited by 11
  17. 2025AnyI2V: Animating Any Conditional Image with Motion Controlcited by 11
  18. 2024CPA: Camera-pose-awareness Diffusion Transformer for Video Generationcited by 10
  19. 2025Frame In-N-Out: Unbounded Controllable Image-to-Video Generationcited by 10
  20. 2025I2V3D: Controllable image-to-video generation with 3D guidancecited by 9
  21. 2025TokenMotion: Decoupled Motion Control via Token Disentanglement for Human-centric Video Generationcited by 9
  22. 2025Training-free Guidance in Text-to-Video Generation via Multimodal Planning and Structured Noise Initializationcited by 9
  23. 2026Goal Force: Teaching Video Models To Accomplish Physics-Conditioned Goalscited by 9
  24. 2026SymphoMotion: Joint Control of Camera Motion and Object Dynamics for Coherent Video Generationcited by 9
  25. 2024Video Creation by Demonstrationcited by 8
  26. 2025ToonComposer: Streamlining Cartoon Production with Generative Post-Keyframingcited by 8
  27. 2026Future Optical Flow Prediction Improves Robot Control & Video Generationcited by 7
  28. 2026HVG-3D: Bridging Real and Simulation Domains for 3D-Conditional Hand-Object Interaction Video Synthesiscited by 7
  29. 2024Reenact Anything: Semantic Video Motion Transfer Using Motion-Textual Inversioncited by 6
  30. 2026FlashMotion: Few-Step Controllable Video Generation with Trajectory Guidancecited by 5
  31. 2025On Equivariance and Fast Sampling in Video Diffusion Models Trained with Warped Noisecited by 4
  32. 2025ReVision: Refining Video Diffusion with Explicit 3D Motion Modelingcited by 4
  33. 2026DreamVideo-Omni: Omni-Motion Controlled Multi-Subject Video Customization with Latent Identity Reinforcement Learningcited by 4
  34. 2025Enabling Versatile Controls for Video Diffusion Modelscited by 3
  35. 2025View-Consistent Diffusion Representations for 3D-Consistent Video Generationcited by 3
  36. 2026Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Jointscited by 3
  37. 2026Robot-Factored World Models via Robot Renderingcited by 3
  38. 2025What If : Understanding Motion Through Sparse Interactionscited by 2

50 most cited of 62 All 62 (JSON)