# Representing motion as a sequence of latent primitives, a flexible approach for human motion modelling

arXiv: https://arxiv.org/abs/2206.13142 · HTML: https://mlsanity.com/paper/2206.13142/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 27 Jun 2022.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [Learning 3D Human Dynamics from Video](https://mlsanity.com/paper/1812.01601/) (2018)
- [Real-time Deep Dynamic Characters](https://mlsanity.com/paper/2105.01794/) (2021)
- [Learning Motion Priors for 4D Human Body Capture in 3D Scenes](https://mlsanity.com/paper/2108.10399/) (2021)
- [Predicting 3D Human Dynamics from Video](https://mlsanity.com/paper/1908.04781/) (2019)
- [CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations](https://mlsanity.com/paper/2008.02792/) (2020)
- [Task-Generic Hierarchical Human Motion Prior using VAEs](https://mlsanity.com/paper/2106.04004/) (2021)
- [Learning Compositional Representation for 4D Captures with Neural ODE](https://mlsanity.com/paper/2103.08271/) (2021)
- [Probabilistic Character Motion Synthesis using a Hierarchical Deep Latent Variable Model](https://mlsanity.com/paper/2010.09950/) (2020)
- [H4D: Human 4D Modeling by Learning Neural Compositional Representation](https://mlsanity.com/paper/2203.01247/) (2022)
- [A structured latent space for human body motion generation](https://mlsanity.com/paper/2106.04387/) (2021)
- [Exploring Versatile Prior for Human Motion via Motion Frequency Guidance](https://mlsanity.com/paper/2111.13074/) (2021)

## Built on by (most cited first, 3 of 3)

- [D-Garment: Physically Grounded Latent Diffusion for Dynamic Garment Deformations](https://mlsanity.com/paper/2504.03468/) (2025)
- [Quality assessment of 3D human animation: Subjective and objective evaluation](https://mlsanity.com/paper/2505.23301/) (2025)
- [Cepheid: Decoupling Scale from Structure for Cross-Family Motion Completion](https://mlsanity.com/paper/2610.18871/) (2026)

## Abstract (arXiv)

We propose a new representation of human body motion which encodes a full motion in a sequence of latent motion primitives. Recently, task generic motion priors have been introduced and propose a coherent representation of human motion based on a single latent code, with encouraging results for many tasks. Extending these methods to longer motion with various duration and framerate is all but straightforward as one latent code proves inefficient to encode longer term variability. Our hypothesis is that long motions are better represented as a succession of actions than in a single block. By leveraging a sequence-to-sequence architecture, we propose a model that simultaneously learns a temporal segmentation of motion and a prior on the motion segments. To provide flexibility with temporal resolution and motion duration, our representation is continuous in time and can be queried for any timestamp. We show experimentally that our method leads to a significant improvement over state-of-the-art motion priors on a spatio-temporal completion task on sparse pointclouds. Code will be made available upon publication.