# MultiAct: Long-Term 3D Human Motion Generation from Multiple Action Labels

arXiv: https://arxiv.org/abs/2212.05897 · HTML: https://mlsanity.com/paper/2212.05897/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 12 Dec 2022.

## Builds on (most cited first, 26 of 26)

- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020)
- [TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts](https://mlsanity.com/paper/2207.01696/) (2022)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [BABEL: Bodies, Action and Behavior with English Labels](https://mlsanity.com/paper/2106.09696/) (2021)
- [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021)
- [Long-term Human Motion Prediction with Scene Context](https://mlsanity.com/paper/2007.03672/) (2020)
- [DLow: Diversifying Latent Flows for Diverse Human Motion Prediction](https://mlsanity.com/paper/2003.08386/) (2020)
- [HP-GAN: Probabilistic 3D human motion prediction via GAN](https://mlsanity.com/paper/1711.09561/) (2017)
- [Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes](https://mlsanity.com/paper/2012.05522/) (2020)
- [We are More than Our Joints: Predicting how 3D Bodies Move](https://mlsanity.com/paper/2012.00619/) (2020)
- [Text2Action: Generative Adversarial Synthesis from Language to Action](https://mlsanity.com/paper/1710.05298/) (2017)
- [Structured Prediction Helps 3D Human Motion Modelling](https://mlsanity.com/paper/1910.09070/) (2019)
- [Scene-aware Generative Network for Human Motion Synthesis](https://mlsanity.com/paper/2105.14804/) (2021)
- [Pose Guided Human Video Generation](https://mlsanity.com/paper/1807.11152/) (2018)
- [Generating Smooth Pose Sequences for Diverse Human Motion Prediction](https://mlsanity.com/paper/2108.08422/) (2021)
- [Single-Shot Motion Completion with Transformer](https://mlsanity.com/paper/2103.00776/) (2021)
- [Deep Video Generation, Prediction and Completion of Human Action Sequences](https://mlsanity.com/paper/1711.08682/) (2017)
- [Perpetual Motion: Generating Unbounded Human Motion](https://mlsanity.com/paper/2007.13886/) (2020)
- [Task-Generic Hierarchical Human Motion Prior using VAEs](https://mlsanity.com/paper/2106.04004/) (2021)
- [Generative Tweening: Long-term Inbetweening of 3D Human Motions](https://mlsanity.com/paper/2005.08891/) (2020)
- [Dynamic Future Net: Diversified Human Motion Generation](https://mlsanity.com/paper/2009.05109/) (2020)
- [MUGL: Large Scale Multi Person Conditional Action Generation with Locomotion](https://mlsanity.com/paper/2110.11460/) (2021)
- [An Experimental Validation and Comparison of Reaching Motion Models for Unconstrained Handovers: Towards Generating Humanlike Motions for Human-Robot Handovers](https://mlsanity.com/paper/2108.12780/) (2021)

## Built on by (most cited first, 30 of 45)

- [TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis](https://mlsanity.com/paper/2305.00976/) (2023)
- [MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model](https://mlsanity.com/paper/2404.19759/) (2024)
- [HumanTOMATO: Text-aligned Whole-body Motion Generation](https://mlsanity.com/paper/2310.12978/) (2023)
- [Human Motion Generation: A Survey](https://mlsanity.com/paper/2307.10894/) (2023)
- [MotionLLM: Understanding Human Behaviors from Human Motions and Videos](https://mlsanity.com/paper/2405.20340/) (2024)
- [EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation](https://mlsanity.com/paper/2312.02256/) (2023)
- [Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction](https://mlsanity.com/paper/2404.00562/) (2024)
- [Seamless Human Motion Composition with Blended Positional Encodings](https://mlsanity.com/paper/2402.15509/) (2024)
- [SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation](https://mlsanity.com/paper/2304.10417/) (2023)
- [InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction](https://mlsanity.com/paper/2403.19652/) (2024)
- [MotionFix: Text-Driven 3D Human Motion Editing](https://mlsanity.com/paper/2408.00712/) (2024)
- [Motion-Agent: A Conversational Framework for Human Motion Generation with LLMs](https://mlsanity.com/paper/2405.17013/) (2024)
- [HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects](https://mlsanity.com/paper/2407.12371/) (2024)
- [M2D2M: Multi-Motion Generation from Text with Discrete Diffusion Models](https://mlsanity.com/paper/2407.14502/) (2024)
- [Synthesizing Long-Term Human Motions with Diffusion Models via Coherent Sampling](https://mlsanity.com/paper/2308.01850/) (2023)
- [KMM: Key Frame Mask Mamba for Extended Motion Generation](https://mlsanity.com/paper/2411.06481/) (2024)
- [Diffgrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Model](https://mlsanity.com/paper/2412.20657/) (2024)
- [T2LM: Long-Term 3D Human Motion Generation from Multiple Sentences](https://mlsanity.com/paper/2406.00636/) (2024)
- [InfiniDreamer: Arbitrarily Long Human Motion Generation via Segment Score Distillation](https://mlsanity.com/paper/2411.18303/) (2024)
- [Text-driven Motion Generation: Overview, Challenges and Directions](https://mlsanity.com/paper/2505.09379/) (2025)
- [Motion Generation: A Survey of Generative Approaches and Benchmarks](https://mlsanity.com/paper/2507.05419/) (2025)
- [FORCE: Physics-aware Human-object Interaction](https://mlsanity.com/paper/2403.11237/) (2024)
- [Autonomous LLM-Enhanced Adversarial Attack for Text-to-Motion](https://mlsanity.com/paper/2408.00352/) (2024)
- [Efficient Text-driven Motion Generation via Latent Consistency Training](https://mlsanity.com/paper/2405.02791/) (2024)
- [AMD:Anatomical Motion Diffusion with Interpretable Motion Decomposition and Fusion](https://mlsanity.com/paper/2312.12763/) (2023)
- [Orientation-Aware Leg Movement Learning for Action-Driven Human Motion Prediction](https://mlsanity.com/paper/2310.14907/) (2023)
- [Infinite Motion: Extended Motion Generation via Long Text Instructions](https://mlsanity.com/paper/2407.08443/) (2024)
- [ContactGen: Contact-Guided Interactive 3D Human Generation for Partners](https://mlsanity.com/paper/2401.17212/) (2024)
- [Shape Conditioned Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2405.06778/) (2024)
- [COLLAGE: Collaborative Human-Agent Interaction Generation using Hierarchical Latent Diffusion and Language Models](https://mlsanity.com/paper/2409.20502/) (2024)

## Abstract (arXiv)

We tackle the problem of generating long-term 3D human motion from multiple action labels. Two main previous approaches, such as action- and motion-conditioned methods, have limitations to solve this problem. The action-conditioned methods generate a sequence of motion from a single action. Hence, it cannot generate long-term motions composed of multiple actions and transitions between actions. Meanwhile, the motion-conditioned methods generate future motions from initial motion. The generated future motions only depend on the past, so they are not controllable by the user's desired actions. We present MultiAct, the first framework to generate long-term 3D human motion from multiple action labels. MultiAct takes account of both action and motion conditions with a unified recurrent generation system. It repetitively takes the previous motion and action label; then, it generates a smooth transition and the motion of the given action. As a result, MultiAct produces realistic long-term motion controlled by the given sequence of multiple action labels. Codes are available here at https://github.com/TaeryungLee/MultiAct_RELEASE.