# Dynamical Deep Generative Latent Modeling of 3D Skeletal Motion

arXiv: https://arxiv.org/abs/2106.10393 · HTML: https://mlsanity.com/paper/2106.10393/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 70 references that are in the map.
- First posted on arXiv 18 Jun 2021.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networks](https://mlsanity.com/paper/1703.07015/) (2017)
- [Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting](https://mlsanity.com/paper/1907.00235/) (2019)
- [Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images](https://mlsanity.com/paper/1506.07365/) (2015)
- [Structured Inference Networks for Nonlinear State Space Models](https://mlsanity.com/paper/1609.09869/) (2016)
- [Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting](https://mlsanity.com/paper/1905.03806/) (2019)
- [Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data](https://mlsanity.com/paper/1605.06432/) (2016)
- [A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning](https://mlsanity.com/paper/1710.05741/) (2017)
- [Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces](https://mlsanity.com/paper/1905.07357/) (2019)
- [Bayesian Temporal Factorization for Multidimensional Time Series Prediction](https://mlsanity.com/paper/1910.06366/) (2019)
- [Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control](https://mlsanity.com/paper/1806.08666/) (2018)
- [Tree-Structured Recurrent Switching Linear Dynamical Systems for Multi-Scale Modeling](https://mlsanity.com/paper/1811.12386/) (2018)
- [A Memory-Network Based Solution for Multivariate Time-Series Forecasting](https://mlsanity.com/paper/1809.02105/) (2018)
- [Dissimilarity-based Sparse Subset Selection](https://mlsanity.com/paper/1407.6810/) (2014)
- [Switching Linear Dynamics for Variational Bayes Filtering](https://mlsanity.com/paper/1905.12434/) (2019)
- [Deep Switching Auto-Regressive Factorization:Application to Time Series Forecasting](https://mlsanity.com/paper/2009.05135/) (2020)
- [Efficient Unsupervised Temporal Segmentation of Motion Data](https://mlsanity.com/paper/1510.06595/) (2015)
- [Deep Markov Spatio-Temporal Factorization](https://mlsanity.com/paper/2003.09779/) (2020)

## Built on by (most cited first, 1 of 1)

- [Equivariant Deep Dynamical Model for Motion Prediction](https://mlsanity.com/paper/2111.01892/) (2021)

## Abstract (arXiv)

In this paper, we propose a Bayesian switching dynamical model for segmentation of 3D pose data over time that uncovers interpretable patterns in the data and is generative. Our model decomposes highly correlated skeleton data into a set of few spatial basis of switching temporal processes in a low-dimensional latent framework. We parameterize these temporal processes with regard to a switching deep vector autoregressive prior in order to accommodate both multimodal and higher-order nonlinear inter-dependencies. This results in a dynamical deep generative latent model that parses the meaningful intrinsic states in the dynamics of 3D pose data using approximate variational inference, and enables a realistic low-level dynamical generation and segmentation of complex skeleton movements. Our experiments on four biological motion data containing bat flight, salsa dance, walking, and golf datasets substantiate superior performance of our model in comparison with the state-of-the-art methods.