mlsanityData to 14 Oct 2026

Unlocking Temporal Generalization in Hamiltonian Video Dynamics Models

arXiv 2607.07763 ↗First posted 8 Jul 2026cs.LGcited by 0

On the map: World models → Latent predictive world models

Abstract (arXiv)

World models are typically trained to predict discrete-time physical dynamics with a fixed step size baked into the model weights, preventing prediction at variable temporal resolutions. This matters for hierarchical planning, sim-to-real transfer, and scientific or game-engine applications that must query the same dynamics at multiple timescales. Hamiltonian Generative Networks (HGN) offer a principled path forward, grounding predictions in a continuous-time energy function that is, in principle, independent of the observation frame rate. In practice, however, their temporal generalization breaks down in non-conservative settings. We show that in externally forced, dissipative environments, HGN rollouts at step sizes beyond the training regime fail due to distinct failure modes, including latent magnitude growth driven by an unconstrained action-force map, and global truncation error accumulation from an under-resolved integrator. We identify a targeted fix for each mechanism and demonstrate stable dynamics prediction at temporal resolutions well outside the training distribution. In a detailed analysis, we recommend several strategies for enabling temporal generalization in continuous-time video generation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 26 of its 36 references that are in the map. First posted on arXiv 8 Jul 2026.

Builds on

26 of 36 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Hierarchical Planning with Latent World Models3 Apr
  2. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels13 Mar
  3. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning7 Nov
  4. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture19 Jan
  5. 2022Deep Hierarchical Planning from Pixels8 Jun

Other references · 21

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2018Neural Ordinary Differential Equationscited by 3,779
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
  5. 2018World Modelscited by 1,513
  6. 2020Mastering Atari with Discrete World Modelscited by 1,007
  7. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  8. 2019Hamiltonian Neural Networkscited by 712
  9. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  10. 2020Lagrangian Neural Networkscited by 398
  11. 2022Message Passing Neural PDE Solverscited by 386
  12. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
9 more
  1. 2024Diffusion Models Are Real-Time Game Enginescited by 296
  2. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  3. 2024Navigation World Modelscited by 272
  4. 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 254
  5. 2019Latent ODEs for Irregularly-Sampled Time Seriescited by 183
  6. 2019Hamiltonian Generative Networkscited by 137
  7. 2021Port-Hamiltonian Neural Networks for Learning Explicit Time-Dependent Dynamical Systemscited by 67
  8. 2025Long-Context State-Space Video World Modelscited by 64
  9. 2019Making Deep Q-learning methods robust to time discretizationcited by 58

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Physically Native World Models: A Hamiltonian Perspective on Generative World Modeling
  2. 2026LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observations
  3. 2026Correcting a learned physical invariant improves world-model rollouts
  4. 2026LeHami: Symplectic Latent Dynamics for World Models
  5. 2026JEPA-x: Cross-Predictive Physics Grounding for Forecastable Latent Dynamics
  6. 2026Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratio
  7. 2026WALL-SS: Scaling Long-horizon World Models via Next-Scale Autoregression
  8. 2026Latent Generative Solvers for Generalizable Long-Term Physics Simulation