mlsanityData to 14 Oct 2026

NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamics

arXiv 2509.21309 ↗First posted 25 Sep 2025cs.CVcited by 41

Not on the map. Why?

Abstract (arXiv)

A primary bottleneck in large-scale text-to-video generation today is physical consistency and controllability. Despite recent advances, state-of-the-art models often produce unrealistic motions, such as objects falling upward, or abrupt changes in velocity and direction. Moreover, these models lack precise parameter control, struggling to generate physically consistent dynamics under different initial conditions. We argue that this fundamental limitation stems from current models learning motion distributions solely from appearance, while lacking an understanding of the underlying dynamics. In this work, we propose NewtonGen, a framework that integrates data-driven synthesis with learnable physical principles. At its core lies trainable Neural Newtonian Dynamics (NND), which can model and predict a variety of Newtonian motions, thereby injecting latent dynamical constraints into the video generation process. By jointly leveraging data priors and dynamical guidance, NewtonGen enables physically consistent video synthesis with precise parameter control. All data and code are available at https://github.com/pandayuanyu/NewtonGen

Key facts. Cited by 41 papers in the map (data through 14 Oct 2026). Builds on 66 of its 80 references that are in the map. First posted on arXiv 25 Sep 2025.

Builds on

66 of 80 references.

References · 66

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2022Scalable Diffusion Models with Transformerscited by 6,054
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2018Neural Ordinary Differential Equationscited by 3,779
  10. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  11. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  12. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
38 more
  1. 2022Video Diffusion Modelscited by 2,305
  2. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  3. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  4. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  5. 2019An Introduction to Variational Autoencoderscited by 1,000
  6. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
  7. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  8. 2019Hamiltonian Neural Networkscited by 712
  9. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  10. 2023PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamicscited by 377
  11. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 368
  12. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 264
  13. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  14. 2019Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learningcited by 212
  15. 2024How Far is Video Generation from World Model: A Physical Law Perspectivecited by 212
  16. 2020Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Predictioncited by 195
  17. 2024PhysDreamer: Physics-Based Interaction with 3D Objects via Video Generationcited by 191
  18. 2024PhysGen: Rigid-Body Physics-Grounded Image-to-Video Generationcited by 174
  19. 2024Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generationcited by 174
  20. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 164
  21. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 156
  22. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 149
  23. 2024Tarsier: Recipes for Training and Evaluating Large Video Description Modelscited by 141
  24. 2025WorldModelBench: Judging Video Generation Models As World Modelscited by 124
  25. 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 117
  26. 2025VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Modelscited by 113
  27. 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 98
  28. 2017Visual Interaction Networkscited by 95
  29. 2023Generative Image Dynamicscited by 94
  30. 2025WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generationcited by 90
  31. 2024PhyT2V: LLM-Guided Iterative Self-Refinement for Physics-Grounded Text-to-Video Generationcited by 80
  32. 2025Intuitive physics understanding emerges from self-supervised pretraining on natural videoscited by 70
  33. 2025PhysGen3D: Crafting a Miniature Interactive World from a Single Imagecited by 66
  34. 2023GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planningcited by 61
  35. 2025VLIPP: Towards Physically Plausible Video Generation with Vision and Language Informed Physical Priorcited by 52
  36. 2025PISA Experiments: Exploring Physics Post-Training for Video Diffusion Models by Watching Stuff Dropcited by 50
  37. 2024MotionCraft: Physics-based Zero-Shot Video Generationcited by 48
  38. 2025Evaluating Newtonian Mechanics in Video Generative Models with Real Physical Systemscited by 48

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 31.

41 papers.

Sort:

By area:

  1. 2026Inference-time Physics Alignment of Video Generative Models with Latent World Modelscited by 37
  2. 2025What about gravity in video generation? Post-Training Newton's Laws with Verifiable Rewardscited by 25
  3. 2026PhysRVG: Physics-Aware Unified Reinforcement Learning for Video Generative Modelscited by 17
  4. 2025LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preferencecited by 16
  5. 2025RULER-Bench: Probing Rule-based Reasoning Abilities of Next-level Video Generation Models for Vision Foundation Intelligencecited by 13
  6. 2026NEWTON: Agentic Planning for Physically Grounded Video Generationcited by 12
  7. 2025ProPhy: Progressive Physical Alignment for Dynamic World Simulationcited by 11
  8. 2026ACWM-Phys: Investigating Generalized Physical Interaction in Action-Conditioned Video World Modelscited by 10
  9. 2026PhysInOne: Visual Physics Learning and Reasoning in One Suitecited by 9
  10. 2026The Pulse of Motion: Measuring Physical Frame Rate from Visual Dynamicscited by 8
  11. 2026PhyRPR: Training-Free Physics-Constrained Video Generationcited by 7
  12. 2025Objects in Generated Videos Are Slower Than They Appear: Models Suffer Sub-Earth Gravity and Don't Know Galileo's Principle...for nowcited by 5
29 more
  1. 2025SeeU: Seeing the Unseen World via 4D Dynamics-aware Generationcited by 5
  2. 2025MoReGen: Multi-Agent Motion-Reasoning Engine for Code-based Text-to-Video Synthesiscited by 5
  3. 2026GS-Agent: Creating 4D Physical Worlds With Generative Simulationcited by 5
  4. 2025Counterfactual World Models via Digital Twin-conditioned Video Diffusioncited by 4
  5. 2026PhysAlign: Physics-Coherent Image-to-Video Generation through Feature and 3D Representation Alignmentcited by 4
  6. 2026LaMo: Self-Supervised Latent Motion Priors for Physical Realism in Video Generationcited by 4
  7. 2026VERTIGO: Visual Preference Optimization for Cinematic Camera Trajectory Generationcited by 3
  8. 2026Phys4D: Fine-Grained Physics-Consistent 4D Modeling from Video Diffusioncited by 2
  9. 2026OptiWorld: Optimal Control for Video World Generation under Physical Constraintscited by 2
  10. 2026PhysEditWorld: A Large-Scale Dataset Toward Physics-Editable World Modelscited by 2
  11. 2026Principia: Relational Physics Tests for Video Modelscited by 2
  12. 2026Why Do Video Diffusion Models Violate Physics? Unveiling the Flaws in Attention Mechanismscited by 2
  13. 2025FlowSteer: Conditioning Flow Field for Consistent Image Restorationcited by 1
  14. 2026Seeking Physics in Diffusion Noisecited by 1
  15. 2026LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observationscited by 1
  16. 2026NewtPhys: Do Foundation Models Understand Newtonian Physics?cited by 1
  17. 2026Physics-Informed Video Generation via Mixture-of-Experts Latent Alignmentcited by 1
  18. 2026NEXUS: Neural Energy Fields for Physically Consistent Contact-Rich 3D Object Dynamicscited by 1
  19. 2025Envision: Embodied Visual Planning via Goal-Imagery Video Diffusioncited by 0
  20. 2026Tora3: Trajectory-Guided Audio-Video Generation with Physical Coherencecited by 0
  21. 2026VIPER: Visual In-Context Physics Reasoning for Physically Plausible Video Generationcited by 0
  22. 2026NewtonGS: Physics-Structured Object-Level Neural Newtonian Dynamics for Gaussian Scene Animationcited by 0
  23. 2026LaGSplat: Inferring Physics-Governed Interactive Simulation from Monocular Video Using Latent Lagrangian Gaussian Splattingcited by 0
  24. 2026PhysFlow: Physics-Aware Optical Flow for Motion Controllable Video Generationcited by 0
  25. 2026CompAdapt: Adaptable Composite Motion Modeling for Physics-Consistent Text-to-Video Generationcited by 0
  26. 2026VideoGen-Agent: Reinforcing Video Generation Agentscited by 0
  27. 2026OneWorld: Learning Consistent Physics Across Actions in World Modelscited by 0
  28. 2026Reimagine Video Dynamicscited by 0
  29. 2026EVEWorld: Physical Evolution Supervision for Embodied World Modelscited by 0