# Real-time Controllable Motion Transition for Characters

arXiv: https://arxiv.org/abs/2205.02540 · HTML: https://mlsanity.com/paper/2205.02540/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 43 references that are in the map.
- First posted on arXiv 5 May 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [Structural-RNN: Deep Learning on Spatio-Temporal Graphs](https://mlsanity.com/paper/1511.05298/) (2015)
- [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021)
- [Character Controllers Using Motion VAEs](https://mlsanity.com/paper/2103.14274/) (2021)
- [GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras](https://mlsanity.com/paper/2112.01524/) (2021)
- [Human Motion Prediction via Spatio-Temporal Inpainting](https://mlsanity.com/paper/1812.05478/) (2018)
- [Convolutional Autoencoders for Human Motion Infilling](https://mlsanity.com/paper/2010.11531/) (2020)
- [Single-Shot Motion Completion with Transformer](https://mlsanity.com/paper/2103.00776/) (2021)
- [Action-Agnostic Human Pose Forecasting](https://mlsanity.com/paper/1810.09676/) (2018)
- [Recurrent Transition Networks for Character Locomotion](https://mlsanity.com/paper/1810.02363/) (2018)
- [Modeling Human Motion with Quaternion-based Neural Networks](https://mlsanity.com/paper/1901.07677/) (2019)
- [Task-Generic Hierarchical Human Motion Prior using VAEs](https://mlsanity.com/paper/2106.04004/) (2021)
- [Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling](https://mlsanity.com/paper/1908.07214/) (2019)
- [Generative Tweening: Long-term Inbetweening of 3D Human Motions](https://mlsanity.com/paper/2005.08891/) (2020)
- [Dynamic Future Net: Diversified Human Motion Generation](https://mlsanity.com/paper/2009.05109/) (2020)

## Built on by (most cited first, 29 of 29)

- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [EDGE: Editable Dance Generation From Music](https://mlsanity.com/paper/2211.10658/) (2022)
- [Synthesizing Diverse Human Motions in 3D Indoor Scenes](https://mlsanity.com/paper/2305.12411/) (2023)
- [SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation](https://mlsanity.com/paper/2304.10417/) (2023)
- [ParCo: Part-Coordinating Text-to-Motion Synthesis](https://mlsanity.com/paper/2403.18512/) (2024)
- [TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesis](https://mlsanity.com/paper/2307.15042/) (2023)
- [PMP: Learning to Physically Interact with Environments using Part-wise Motion Priors](https://mlsanity.com/paper/2305.03249/) (2023)
- [Plan, Posture and Go: Towards Open-World Text-to-Motion Generation](https://mlsanity.com/paper/2312.14828/) (2023)
- [Motion In-Betweening with Phase Manifolds](https://mlsanity.com/paper/2308.12751/) (2023)
- [M2D2M: Multi-Motion Generation from Text with Discrete Diffusion Models](https://mlsanity.com/paper/2407.14502/) (2024)
- [OmniMotionGPT: Animal Motion Generation with Limited Data](https://mlsanity.com/paper/2311.18303/) (2023)
- [RSMT: Real-time Stylized Motion Transition for Characters](https://mlsanity.com/paper/2306.11970/) (2023)
- [Pay Attention and Move Better: Harnessing Attention for Interactive Motion Generation and Training-free Editing](https://mlsanity.com/paper/2410.18977/) (2024)
- [Continuous Intermediate Token Learning with Implicit Motion Manifold for Keyframe Based Motion Interpolation](https://mlsanity.com/paper/2303.14926/) (2023)
- [Hard No-Box Adversarial Attack on Skeleton-Based Human Action Recognition with Skeleton-Motion-Informed Gradient](https://mlsanity.com/paper/2308.05681/) (2023)
- [T2M-HiFiGPT: Generating High Quality Human Motion from Textual Descriptions with Residual Discrete Representations](https://mlsanity.com/paper/2312.10628/) (2023)
- [SemanticBoost: Elevating Motion Generation with Augmented Textual Cues](https://mlsanity.com/paper/2310.20323/) (2023)
- [TASAR: Transfer-based Attack on Skeletal Action Recognition](https://mlsanity.com/paper/2409.02483/) (2024)
- [Human Motion Prediction under Unexpected Perturbation](https://mlsanity.com/paper/2403.15891/) (2024)
- [Decoupling Contact for Fine-Grained Motion Style Transfer](https://mlsanity.com/paper/2409.05387/) (2024)
- [T2M-X: Learning Expressive Text-to-Motion Generation from Partially Annotated Data](https://mlsanity.com/paper/2409.13251/) (2024)
- [Real-time Diverse Motion In-betweening with Space-time Control](https://mlsanity.com/paper/2410.00270/) (2024)
- [AnyMoLe: Any Character Motion In-betweening Leveraging Video Diffusion Models](https://mlsanity.com/paper/2503.08417/) (2025)
- [Towards Synthesized and Editable Motion In-Betweening Through Part-Wise Phase Representation](https://mlsanity.com/paper/2503.08180/) (2025)
- [Diverse Motion In-betweening with Dual Posture Stitching](https://mlsanity.com/paper/2303.14457/) (2023)
- [SILK: Smooth InterpoLation frameworK for motion in-betweening A Simplified Computational Approach](https://mlsanity.com/paper/2506.09075/) (2025)
- [A Plug-and-Play Multi-Criteria Guidance for Diverse In-Betweening Human Motion Generation](https://mlsanity.com/paper/2508.01590/) (2025)
- [Boosting Adversarial Transferability for Skeleton-based Action Recognition via Exploring the Model Posterior Space](https://mlsanity.com/paper/2407.08572/) (2024)
- [Motion In-Betweening for Densely Interacting Characters](https://mlsanity.com/paper/2510.00314/) (2025)

## Abstract (arXiv)

Real-time in-between motion generation is universally required in games and highly desirable in existing animation pipelines. Its core challenge lies in the need to satisfy three critical conditions simultaneously: quality, controllability and speed, which renders any methods that need offline computation (or post-processing) or cannot incorporate (often unpredictable) user control undesirable. To this end, we propose a new real-time transition method to address the aforementioned challenges. Our approach consists of two key components: motion manifold and conditional transitioning. The former learns the important low-level motion features and their dynamics; while the latter synthesizes transitions conditioned on a target frame and the desired transition duration. We first learn a motion manifold that explicitly models the intrinsic transition stochasticity in human motions via a multi-modal mapping mechanism. Then, during generation, we design a transition model which is essentially a sampling strategy to sample from the learned manifold, based on the target frame and the aimed transition duration. We validate our method on different datasets in tasks where no post-processing or offline computation is allowed. Through exhaustive evaluation and comparison, we show that our method is able to generate high-quality motions measured under multiple metrics. Our method is also robust under various target frames (with extreme cases).