mlsanityData to 14 Oct 2026

SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision

arXiv 2111.05986 ↗First posted 10 Nov 2021stat.ML cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

A recently proposed class of models attempts to learn latent dynamics from high-dimensional observations, like images, using priors informed by Hamiltonian mechanics. While these models have important potential applications in areas like robotics or autonomous driving, there is currently no good way to evaluate their performance: existing methods primarily rely on image reconstruction quality, which does not always reflect the quality of the learnt latent dynamics. In this work, we empirically highlight the problems with the existing measures and develop a set of new measures, including a binary indicator of whether the underlying Hamiltonian dynamics have been faithfully captured, which we call Symplecticity Metric or SyMetric. Our measures take advantage of the known properties of Hamiltonian dynamics and are more discriminative of the model's ability to capture the underlying dynamics than reconstruction error. Using SyMetric, we identify a set of architectural choices that significantly improve the performance of a previously proposed model for inferring latent dynamics from pixels, the Hamiltonian Generative Network (HGN). Unlike the original HGN, the new HGN++ is able to discover an interpretable phase space with physically meaningful latents on some datasets. Furthermore, it is stable for significantly longer rollouts on a diverse range of 13 datasets, producing rollouts of essentially infinite length both forward and backwards in time with no degradation in quality on a subset of the datasets.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 38 of its 59 references that are in the map. First posted on arXiv 10 Nov 2021.

Builds on

38 of 59 references.

References · 38

  1. 2019Hamiltonian Neural Networkscited by 712
  2. 2018Which Training Methods for GANs do actually Converge?cited by 679
  3. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  4. 2020Lagrangian Neural Networkscited by 398
  5. 2016Generating Images with Perceptual Similarity Metrics based on Deep Networkscited by 222
  6. 2019Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learningcited by 212
  7. 2016Unrolled Generative Adversarial Networkscited by 208
  8. 2019Symplectic ODE-Net: Learning Hamiltonian Dynamics with Controlcited by 184
  9. 2017Searching for Activation Functionscited by 169
  10. 2019Symplectic Recurrent Neural Networkscited by 150
  11. 2019Hamiltonian Generative Networkscited by 137
  12. 2018The Mechanics of n-Player Differentiable Gamescited by 118
26 more
  1. 2019Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEscited by 117
  2. 2020Forecasting Sequential Data using Consistent Koopman Autoencoderscited by 115
  3. 2017The Numerics of GANscited by 105
  4. 2020Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraintscited by 105
  5. 2018Taming VAEscited by 85
  6. 2020Dissipative SymODEN: Encoding Hamiltonian Dynamics with Dissipation and Control into Deep Learningcited by 79
  7. 2020Visual Grounding of Learned Physical Modelscited by 55
  8. 2019Unsupervised Model Selection for Variational Disentangled Representation Learningcited by 46
  9. 2020Benchmarking Energy-Conserving Neural Networks for Learning Dynamics from Datacited by 40
  10. 2019Equivariant Hamiltonian Flowscited by 39
  11. 2020Unsupervised Learning of Lagrangian Dynamics from Images for Prediction and Controlcited by 38
  12. 2019Variational Integrator Networks for Physically Structured Embeddingscited by 36
  13. 2020Modeling System Dynamics with Physics-Informed Neural Networks Based on Lagrangian Mechanicscited by 32
  14. 2020Nonseparable Symplectic Neural Networkscited by 31
  15. 2021Which priors matter? Benchmarking models for learning latent dynamicscited by 26
  16. 2020Symplectic Neural Networks in Taylor Series Form for Hamiltonian Systemscited by 24
  17. 2020LagNetViP: A Lagrangian Neural Network for Video Predictioncited by 24
  18. 2020Symbolic Pregression: Discovering Physical Laws from Distorted Videocited by 21
  19. 2020Training Generative Adversarial Networks by Solving Ordinary Differential Equationscited by 21
  20. 2020Sparse Symplectically Integrated Neural Networkscited by 20
  21. 2020Time-Reversal Symmetric ODE Networkcited by 19
  22. 2020Symplectic Gaussian Process Regression of Hamiltonian Flow Mapscited by 17
  23. 2020Forecasting Hamiltonian dynamics without canonical coordinatescited by 17
  24. 2019Multi-Agent Learning in Network Zero-Sum Games is a Hamiltonian Systemcited by 13
  25. 2020Variational Integrator Graph Networks for Learning Energy Conserving Dynamical Systemscited by 7
  26. 2019Rodent: Relevance determination in differential equationscited by 4

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 1.

5 papers.

  1. 2022Continuous MDP Homomorphisms and Homomorphic Policy Gradientcited by 27
  2. 2022Symmetry-Based Representations for Artificial and Biological General Intelligencecited by 26
  3. 2023Policy Gradient Methods in the Presence of Symmetries and State Abstractionscited by 9
  4. 2023Hamiltonian GANcited by 4
  5. 2024Unsupervised Learning of Hybrid Latent Dynamics: A Learn-to-Identify Frameworkcited by 0