Analysis of ODE2VAE with Examples

Deep generative models aim to learn underlying distributions that generate the observed data. Given the fact that the generative distribution may be complex and intractable, deep latent variable models use probabilistic frameworks to learn more expressive joint probability distributions over the data and their low-dimensional hidden variables. Learning complex probability distributions over sequential data without any supervision is a difficult task for deep generative models. Ordinary Differential Equation Variational Auto-Encoder (ODE2VAE) is a deep latent variable model that aims to learn complex distributions over high-dimensional sequential data and their low-dimensional representations. ODE2VAE infers continuous latent dynamics of the high-dimensional input in a low-dimensional hierarchical latent space. The hierarchical organization of the continuous latent space embeds a physics-guided inductive bias in the model. In this paper, we analyze the latent representations inferred by the ODE2VAE model over three different physical motion datasets: bouncing balls, projectile motion, and simple pendulum. Through our experiments, we explore the effects of the physics-guided inductive bias of the ODE2VAE model over the learned dynamical latent representations. We show that the model is able to learn meaningful latent representations to an extent without any supervision.