MoRight: Motion Control Done Right
Not on the map. Why?
Abstract (arXiv)
Generating motion-controlled videos--where user-specified actions drive physically plausible scene dynamics under freely chosen viewpoints--demands two capabilities: (1) disentangled motion control, allowing users to separately control the object motion and adjust camera viewpoint; and (2) motion causality, ensuring that user-driven actions trigger coherent reactions from other objects rather than merely displacing pixels. Existing methods fall short on both fronts: they entangle camera and object motion into a single tracking signal and treat motion as kinematic displacement without modeling causal relationships between object motion. We introduce MoRight, a unified framework that addresses both limitations through disentangled motion modeling. Object motion is specified in a canonical static-view and transferred to an arbitrary target camera viewpoint via temporal cross-view attention, enabling disentangled camera and object control. We further decompose motion into active (user-driven) and passive (consequence) components, training the model to learn motion causality from data. At inference, users can either supply active motion and MoRight predicts consequences (forward reasoning), or specify desired passive outcomes and MoRight recovers plausible driving actions (inverse reasoning), all while freely adjusting the camera viewpoint. Experiments on three benchmarks demonstrate state-of-the-art performance in generation quality, motion controllability, and interaction awareness.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 85 of its 94 references that are in the map. First posted on arXiv 8 Apr 2026.
Builds on
85 of 94 references.
References · 85
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
38 more
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
- 2020First Order Motion Model for Image Animationcited by 698
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
- 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
- 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
- 2023CoTracker: It is Better to Track Togethercited by 516
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
- 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 441
- 2023Learning Interactive Real-World Simulatorscited by 440
- 2026World Action Models are Zero-shot Policiescited by 411
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
- 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 389
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
- 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
- 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 281
- 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
- 2016Deep Visual Foresight for Planning Robot Motioncited by 276
- 2024Navigation World Modelscited by 262
- 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
- 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 224
- 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 215
- 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
- 2020Robust Consistent Video Depth Estimationcited by 188
- 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 185
- 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
- 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 176
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 174
- 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 169
- 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
- 2024DragAnything: Motion Control for Anything using Entity Representationcited by 159
- 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 141
- 2024VLP: Vision Language Planning for Autonomous Drivingcited by 125
50 most cited of 85 All 85 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.