# SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision

arXiv: https://arxiv.org/abs/2111.05986 · HTML: https://mlsanity.com/paper/2111.05986/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 59 references that are in the map.
- First posted on arXiv 10 Nov 2021.

## Builds on (most cited first, 30 of 38)

- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Lagrangian Neural Networks](https://mlsanity.com/paper/2003.04630/) (2020)
- [Generating Images with Perceptual Similarity Metrics based on Deep Networks](https://mlsanity.com/paper/1602.02644/) (2016)
- [Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning](https://mlsanity.com/paper/1907.04490/) (2019)
- [Unrolled Generative Adversarial Networks](https://mlsanity.com/paper/1611.02163/) (2016)
- [Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control](https://mlsanity.com/paper/1909.12077/) (2019)
- [Searching for Activation Functions](https://mlsanity.com/paper/1710.05941/) (2017)
- [Symplectic Recurrent Neural Networks](https://mlsanity.com/paper/1909.13334/) (2019)
- [Hamiltonian Generative Networks](https://mlsanity.com/paper/1909.13789/) (2019)
- [The Mechanics of n-Player Differentiable Games](https://mlsanity.com/paper/1802.05642/) (2018)
- [Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs](https://mlsanity.com/paper/1901.07017/) (2019)
- [Forecasting Sequential Data using Consistent Koopman Autoencoders](https://mlsanity.com/paper/2003.02236/) (2020)
- [The Numerics of GANs](https://mlsanity.com/paper/1705.10461/) (2017)
- [Simplifying Hamiltonian and Lagrangian Neural Networks via Explicit Constraints](https://mlsanity.com/paper/2010.13581/) (2020)
- [Taming VAEs](https://mlsanity.com/paper/1810.00597/) (2018)
- [Dissipative SymODEN: Encoding Hamiltonian Dynamics with Dissipation and Control into Deep Learning](https://mlsanity.com/paper/2002.08860/) (2020)
- [Visual Grounding of Learned Physical Models](https://mlsanity.com/paper/2004.13664/) (2020)
- [Unsupervised Model Selection for Variational Disentangled Representation Learning](https://mlsanity.com/paper/1905.12614/) (2019)
- [Benchmarking Energy-Conserving Neural Networks for Learning Dynamics from Data](https://mlsanity.com/paper/2012.02334/) (2020)
- [Equivariant Hamiltonian Flows](https://mlsanity.com/paper/1909.13739/) (2019)
- [Unsupervised Learning of Lagrangian Dynamics from Images for Prediction and Control](https://mlsanity.com/paper/2007.01926/) (2020)
- [Variational Integrator Networks for Physically Structured Embeddings](https://mlsanity.com/paper/1910.09349/) (2019)
- [Modeling System Dynamics with Physics-Informed Neural Networks Based on Lagrangian Mechanics](https://mlsanity.com/paper/2005.14617/) (2020)
- [Nonseparable Symplectic Neural Networks](https://mlsanity.com/paper/2010.12636/) (2020)
- [Which priors matter? Benchmarking models for learning latent dynamics](https://mlsanity.com/paper/2111.05458/) (2021)
- [Symplectic Neural Networks in Taylor Series Form for Hamiltonian Systems](https://mlsanity.com/paper/2005.04986/) (2020)
- [LagNetViP: A Lagrangian Neural Network for Video Prediction](https://mlsanity.com/paper/2010.12932/) (2020)
- [Symbolic Pregression: Discovering Physical Laws from Distorted Video](https://mlsanity.com/paper/2005.11212/) (2020)

## Built on by (most cited first, 5 of 5)

- [Continuous MDP Homomorphisms and Homomorphic Policy Gradient](https://mlsanity.com/paper/2209.07364/) (2022)
- [Symmetry-Based Representations for Artificial and Biological General Intelligence](https://mlsanity.com/paper/2203.09250/) (2022)
- [Policy Gradient Methods in the Presence of Symmetries and State Abstractions](https://mlsanity.com/paper/2305.05666/) (2023)
- [Hamiltonian GAN](https://mlsanity.com/paper/2308.11216/) (2023)
- [Unsupervised Learning of Hybrid Latent Dynamics: A Learn-to-Identify Framework](https://mlsanity.com/paper/2403.08194/) (2024)

## Abstract (arXiv)

A recently proposed class of models attempts to learn latent dynamics from high-dimensional observations, like images, using priors informed by Hamiltonian mechanics. While these models have important potential applications in areas like robotics or autonomous driving, there is currently no good way to evaluate their performance: existing methods primarily rely on image reconstruction quality, which does not always reflect the quality of the learnt latent dynamics. In this work, we empirically highlight the problems with the existing measures and develop a set of new measures, including a binary indicator of whether the underlying Hamiltonian dynamics have been faithfully captured, which we call Symplecticity Metric or SyMetric. Our measures take advantage of the known properties of Hamiltonian dynamics and are more discriminative of the model's ability to capture the underlying dynamics than reconstruction error. Using SyMetric, we identify a set of architectural choices that significantly improve the performance of a previously proposed model for inferring latent dynamics from pixels, the Hamiltonian Generative Network (HGN). Unlike the original HGN, the new HGN++ is able to discover an interpretable phase space with physically meaningful latents on some datasets. Furthermore, it is stable for significantly longer rollouts on a diverse range of 13 datasets, producing rollouts of essentially infinite length both forward and backwards in time with no degradation in quality on a subset of the datasets.