# LAMP: Language-Assisted Motion Planning for Controllable Video Generation

arXiv: https://arxiv.org/abs/2512.03619 · HTML: https://mlsanity.com/paper/2512.03619/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 36 references that are in the map.
- First posted on arXiv 3 Dec 2025.

## Builds on (most cited first, 29 of 29)

- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control](https://mlsanity.com/paper/2503.03751/) (2025)
- [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023)
- [ReCamMaster: Camera-Controlled Generative Rendering from A Single Video](https://mlsanity.com/paper/2503.11647/) (2025)
- [Video Depth Anything: Consistent Depth Estimation for Super-Long Videos](https://mlsanity.com/paper/2501.12375/) (2025)
- [Stable Virtual Camera: Generative View Synthesis with Diffusion Models](https://mlsanity.com/paper/2503.14489/) (2025)
- [Motion Prompting: Controlling Video Generation with Motion Trajectories](https://mlsanity.com/paper/2412.02700/) (2024)
- [VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning](https://mlsanity.com/paper/2309.15091/) (2023)
- [CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models](https://mlsanity.com/paper/2503.10592/) (2025)
- [CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generation](https://mlsanity.com/paper/2502.08639/) (2025)
- [Towards Understanding Camera Motions in Any Video](https://mlsanity.com/paper/2504.15376/) (2025)
- [RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Control](https://mlsanity.com/paper/2502.10059/) (2025)
- [Director3D: Real-world Camera Trajectory and 3D Scene Generation from Text](https://mlsanity.com/paper/2406.17601/) (2024)
- [CamCloneMaster: Enabling Reference-based Camera Control for Video Generation](https://mlsanity.com/paper/2506.03140/) (2025)
- [FloVD: Optical Flow Meets Video Diffusion Model for Enhanced Camera-Controlled Video Synthesis](https://mlsanity.com/paper/2502.08244/) (2025)
- [E.T. the Exceptional Trajectories: Text-to-camera-trajectory generation with character awareness](https://mlsanity.com/paper/2407.01516/) (2024)
- [VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generation](https://mlsanity.com/paper/2502.07531/) (2025)
- [EPiC: Efficient Video Camera Control Learning with Precise Anchor-Video Guidance](https://mlsanity.com/paper/2505.21876/) (2025)
- [I2VControl: Disentangled and Unified Video Motion Synthesis Control](https://mlsanity.com/paper/2411.17765/) (2024)
- [Cinematographic Camera Diffusion Model](https://mlsanity.com/paper/2402.16143/) (2024)
- [GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photography](https://mlsanity.com/paper/2504.07083/) (2025)
- [RoPECraft: Training-Free Motion Transfer with Trajectory-Guided RoPE Optimization on Diffusion Transformers](https://mlsanity.com/paper/2505.13344/) (2025)
- [Modular-Cam: Modular Dynamic Camera-view Video Generation with LLM](https://mlsanity.com/paper/2504.12048/) (2025)
- [LightMotion: A Light and Tuning-free Method for Simulating Camera Motion in Video Generation](https://mlsanity.com/paper/2503.06508/) (2025)
- [SAGE: Structure-Aware Generative Video Transitions between Diverse Clips](https://mlsanity.com/paper/2510.24667/) (2025)

## Built on by (most cited first, 4 of 4)

- [VII: Visual Instruction Injection for Jailbreaking Image-to-Video Generation Models](https://mlsanity.com/paper/2602.20999/) (2026)
- [LooseControlVideo: Directorial Video Control using Spatial Blocking](https://mlsanity.com/paper/2606.19495/) (2026)
- [MotiMotion: Motion-Controlled Video Generation with Visual Reasoning](https://mlsanity.com/paper/2605.22818/) (2026)
- [CinemaTraj: Composing Atomic Camera Trajectories for 3D Scenes with LLM Agents](https://mlsanity.com/paper/2607.26910/) (2026)

## Abstract (arXiv)

Video generation has achieved remarkable progress in visual fidelity and controllability, enabling conditioning on text, layout, or motion. Among these, motion control - specifying object dynamics and camera trajectories - is essential for composing complex, cinematic scenes, yet existing interfaces remain limited. We introduce LAMP that leverages large language models (LLMs) as motion planners to translate natural language descriptions into explicit 3D trajectories for dynamic objects and (relatively defined) cameras. LAMP defines a motion domain-specific language (DSL), inspired by cinematography conventions. By harnessing program synthesis capabilities of LLMs, LAMP generates structured motion programs from natural language, which are deterministically mapped to 3D trajectories. We construct a large-scale procedural dataset pairing natural text descriptions with corresponding motion programs and 3D trajectories. Experiments demonstrate LAMP's improved performance in motion controllability and alignment with user intent compared to state-of-the-art alternatives establishing the first framework for generating both object and camera motions directly from natural language specifications. Code, models and data are available on our project page.