# Diverse Motion In-betweening with Dual Posture Stitching

arXiv: https://arxiv.org/abs/2303.14457 · HTML: https://mlsanity.com/paper/2303.14457/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 25 Mar 2023.

## Builds on (most cited first, 26 of 26)

- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Structural-RNN: Deep Learning on Spatio-Temporal Graphs](https://mlsanity.com/paper/1511.05298/) (2015)
- [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021)
- [Long-term Human Motion Prediction with Scene Context](https://mlsanity.com/paper/2007.03672/) (2020)
- [We are More than Our Joints: Predicting how 3D Bodies Move](https://mlsanity.com/paper/2012.00619/) (2020)
- [Human Motion Prediction via Spatio-Temporal Inpainting](https://mlsanity.com/paper/1812.05478/) (2018)
- [QuaterNet: A Quaternion-based Recurrent Model for Human Motion](https://mlsanity.com/paper/1805.06485/) (2018)
- [Structured Prediction Helps 3D Human Motion Modelling](https://mlsanity.com/paper/1910.09070/) (2019)
- [The Pose Knows: Video Forecasting by Generating Pose Futures](https://mlsanity.com/paper/1705.00053/) (2017)
- [A Neural Temporal Model for Human Motion Prediction](https://mlsanity.com/paper/1809.03036/) (2018)
- [Convolutional Autoencoders for Human Motion Infilling](https://mlsanity.com/paper/2010.11531/) (2020)
- [BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN](https://mlsanity.com/paper/1812.02591/) (2018)
- [Deep Video Generation, Prediction and Completion of Human Action Sequences](https://mlsanity.com/paper/1711.08682/) (2017)
- [Recurrent Transition Networks for Character Locomotion](https://mlsanity.com/paper/1810.02363/) (2018)
- [Human Motion Modeling using DVGANs](https://mlsanity.com/paper/1804.10652/) (2018)
- [LoBSTr: Real-time Lower-body Pose Prediction from Sparse Upper-body Tracking Signals](https://mlsanity.com/paper/2103.01500/) (2021)
- [Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control](https://mlsanity.com/paper/1806.08666/) (2018)
- [Real-time Controllable Motion Transition for Characters](https://mlsanity.com/paper/2205.02540/) (2022)
- [Conditional Motion In-betweening](https://mlsanity.com/paper/2202.04307/) (2022)
- [Hierarchical Style-based Networks for Motion Synthesis](https://mlsanity.com/paper/2008.10162/) (2020)
- [PVRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction](https://mlsanity.com/paper/1906.06514/) (2019)
- [Probabilistic Character Motion Synthesis using a Hierarchical Deep Latent Variable Model](https://mlsanity.com/paper/2010.09950/) (2020)
- [TrajeVAE: Controllable Human Motion Generation from Trajectories](https://mlsanity.com/paper/2104.00351/) (2021)
- [Human Motion Prediction via Pattern Completion in Latent Representation Space](https://mlsanity.com/paper/1904.09039/) (2019)

## Built on by (most cited first, 2 of 2)

- [Seamless Human Motion Composition with Blended Positional Encodings](https://mlsanity.com/paper/2402.15509/) (2024)
- [Orientation-Aware Leg Movement Learning for Action-Driven Human Motion Prediction](https://mlsanity.com/paper/2310.14907/) (2023)

## Abstract (arXiv)

In-betweening is a technique for generating transitions given initial and target character states. The majority of existing works require multiple (often >10) frames as input, which are not always accessible. Our work deals with a focused yet challenging problem: to generate the transition when given exactly two frames (only the first and last). To cope with this challenging scenario, we implement our bi-directional scheme which generates forward and backward transitions from the start and end frames with two adversarial autoregressive networks, and stitches them in the middle of the transition where there is no strict ground truth. The autoregressive networks based on conditional variational autoencoders (CVAE) are optimized by searching for a pair of optimal latent codes that minimize a novel stitching loss between their outputs. Results show that our method achieves higher motion quality and more diverse results than existing methods on both the LaFAN1 and Human3.6m datasets.