# Generative Video Motion Editing with 3D Point Tracks

arXiv: https://arxiv.org/abs/2512.02015 · HTML: https://mlsanity.com/paper/2512.02015/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 133 of its 139 references that are in the map.
- First posted on arXiv 1 Dec 2025.

## Builds on (most cited first, 30 of 133)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators](https://mlsanity.com/paper/2303.13439/) (2023)
- [Movie Gen: A Cast of Media Foundation Models](https://mlsanity.com/paper/2410.13720/) (2024)
- [MotionCtrl: A Unified and Flexible Motion Controller for Video Generation](https://mlsanity.com/paper/2312.03641/) (2023)
- [Continuous 3D Perception Model with Persistent State](https://mlsanity.com/paper/2501.12387/) (2025)
- [DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision](https://mlsanity.com/paper/2312.16256/) (2023)
- [VideoCrafter1: Open Diffusion Models for High-Quality Video Generation](https://mlsanity.com/paper/2310.19512/) (2023)
- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors](https://mlsanity.com/paper/2310.12190/) (2023)
- [FateZero: Fusing Attentions for Zero-shot Text-based Video Editing](https://mlsanity.com/paper/2303.09535/) (2023)

## Built on by (most cited first, 23 of 23)

- [4RC: 4D Reconstruction via Conditional Querying Anytime and Anywhere](https://mlsanity.com/paper/2602.10094/) (2026)
- [FreeOrbit4D: Training-Free Arbitrary Camera Redirection for Monocular Videos via Foreground-Complete 4D Reconstruction](https://mlsanity.com/paper/2601.18993/) (2026)
- [SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformers](https://mlsanity.com/paper/2606.27345/) (2026)
- [Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Joints](https://mlsanity.com/paper/2603.11755/) (2026)
- [WorldCraft: From Camera Navigation to Object Manipulation in Interactive Video World Models](https://mlsanity.com/paper/2605.25077/) (2026)
- [OptiWorld: Optimal Control for Video World Generation under Physical Constraints](https://mlsanity.com/paper/2606.00499/) (2026)
- [Track2View: 4D-Consistent Camera-Controlled Video Generation via Paired 3D Point Tracks](https://mlsanity.com/paper/2606.15534/) (2026)
- [LooseControlVideo: Directorial Video Control using Spatial Blocking](https://mlsanity.com/paper/2606.19495/) (2026)
- [Lift4D: Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wild](https://mlsanity.com/paper/2606.23688/) (2026)
- [TokenDial: Continuous Attribute Control for Text-to-Video Generation in Visual Dial Space](https://mlsanity.com/paper/2603.27520/) (2026)
- [TrajectoryMover: Generative Movement of Object Trajectories in Videos](https://mlsanity.com/paper/2603.29092/) (2026)
- [Preserve, Reveal, Expand: Faithful 4D Video Editing with Region-Aware Conditioning](https://mlsanity.com/paper/2605.20961/) (2026)
- [MotiMotion: Motion-Controlled Video Generation with Visual Reasoning](https://mlsanity.com/paper/2605.22818/) (2026)
- [TrAction: Action Recognition with Sparse Trajectories](https://mlsanity.com/paper/2606.03490/) (2026)
- [Track the Noise, Move the World:3D-Grounded Motion-Consistent Noise for Controllable Video Generation](https://mlsanity.com/paper/2607.02798/) (2026)
- [Bunraku: Turning a Single Illustration into an Editable Live2D Character](https://mlsanity.com/paper/2607.27348/) (2026)
- [A Study of the Design Space of Motion and Disocclusion Control in Video Generation](https://mlsanity.com/paper/2512.13392/) (2025)
- [AlbedoEdit: Unified Instance-Level Video Editing with Albedo Guidance](https://mlsanity.com/paper/2606.01362/) (2026)
- [Effective Multi-sensor Conditioning for Street-view Novel-view Synthesis](https://mlsanity.com/paper/2606.01590/) (2026)
- [ScaleVid: Geometry-Aware Video Object Scaling with Mesh-Free Inference](https://mlsanity.com/paper/2608.12232/) (2026)
- [4DStreamCtrl: Interactive Video Generation with Online 4D Control](https://mlsanity.com/paper/2608.25479/) (2026)
- [Video Generation Models: A Survey of Post-Training and Alignment](https://mlsanity.com/paper/2610.00812/) (2026)
- [Generative Cinematographer: Composing Camera and Object Motion in 3D](https://mlsanity.com/paper/2610.02180/) (2026)

## Abstract (arXiv)

Camera and object motions are central to a video's narrative. However, precisely editing these captured motions remains a significant challenge, especially under complex object movements. Current motion-controlled image-to-video (I2V) approaches often lack full-scene context for consistent video editing, while video-to-video (V2V) methods provide viewpoint changes or basic object translation, but offer limited control over fine-grained object motion. We present a track-conditioned V2V framework that enables joint editing of camera and object motion. We achieve this by conditioning a video generation model on a source video and paired 3D point tracks representing source and target motions. These 3D tracks establish sparse correspondences that transfer rich context from the source video to new motions while preserving spatiotemporal coherence. Crucially, compared to 2D tracks, 3D tracks provide explicit depth cues, allowing the model to resolve depth order and handle occlusions for precise motion editing. Trained in two stages on synthetic and real data, our model supports diverse motion edits, including joint camera/object manipulation, motion transfer, and non-rigid deformation, unlocking new creative potential in video editing.