# T2LM: Long-Term 3D Human Motion Generation from Multiple Sentences arXiv: https://arxiv.org/abs/2406.00636 ยท HTML: https://mlsanity.com/paper/2406.00636/ ## Key facts - Cited by 12 papers in the map (data through 14 Oct 2026). - Builds on 47 of its 63 references that are in the map. - First posted on arXiv 2 Jun 2024. ## Builds on (most cited first, 30 of 47) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017) - [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013) - [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019) - [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022) - [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022) - [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023) - [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021) - [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022) - [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021) - [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020) - [MotionCLIP: Exposing Human Motion Generation to CLIP Space](https://mlsanity.com/paper/2203.08063/) (2022) - [PhysDiff: Physics-Guided Human Motion Diffusion Model](https://mlsanity.com/paper/2212.02500/) (2022) - [TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts](https://mlsanity.com/paper/2207.01696/) (2022) - [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023) - [EDGE: Editable Dance Generation From Music](https://mlsanity.com/paper/2211.10658/) (2022) - [BABEL: Bodies, Action and Behavior with English Labels](https://mlsanity.com/paper/2106.09696/) (2021) - [Language2Pose: Natural Language Grounded Pose Forecasting](https://mlsanity.com/paper/1907.01108/) (2019) - [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021) - [Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory](https://mlsanity.com/paper/2203.13055/) (2022) - [Learning Individual Styles of Conversational Gesture](https://mlsanity.com/paper/1906.04160/) (2019) - [DLow: Diversifying Latent Flows for Diverse Human Motion Prediction](https://mlsanity.com/paper/2003.08386/) (2020) - [Synthesis of Compositional Animations from Textual Descriptions](https://mlsanity.com/paper/2103.14675/) (2021) - [TEACH: Temporal Action Composition for 3D Humans](https://mlsanity.com/paper/2209.04066/) (2022) - [We are More than Our Joints: Predicting how 3D Bodies Move](https://mlsanity.com/paper/2012.00619/) (2020) - [QuaterNet: A Quaternion-based Recurrent Model for Human Motion](https://mlsanity.com/paper/1805.06485/) (2018) - [Text2Action: Generative Adversarial Synthesis from Language to Action](https://mlsanity.com/paper/1710.05298/) (2017) - [Structured Prediction Helps 3D Human Motion Modelling](https://mlsanity.com/paper/1910.09070/) (2019) - [Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows](https://mlsanity.com/paper/2003.10350/) (2020) - [Learning to Generate Diverse Dance Motions with Transformer](https://mlsanity.com/paper/2008.08171/) (2020) ## Built on by (most cited first, 12 of 12) - [Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation](https://mlsanity.com/paper/2502.20370/) (2025) - [KMM: Key Frame Mask Mamba for Extended Motion Generation](https://mlsanity.com/paper/2411.06481/) (2024) - [InfiniDreamer: Arbitrarily Long Human Motion Generation via Segment Score Distillation](https://mlsanity.com/paper/2411.18303/) (2024) - [PINO: Person-Interaction Noise Optimization for Long-Duration and Customizable Motion Generation of Arbitrary-Sized Groups](https://mlsanity.com/paper/2507.19292/) (2025) - [Text-driven Motion Generation: Overview, Challenges and Directions](https://mlsanity.com/paper/2505.09379/) (2025) - [CASIM: Composite Aware Semantic Injection for Text to Motion Generation](https://mlsanity.com/paper/2502.02063/) (2025) - [E-React: Towards Emotionally Controlled Synthesis of Human Reactions](https://mlsanity.com/paper/2508.06093/) (2025) - [SIGNER: Temporally Grounded Sign Language Generation via Time-Resolved Conditioning](https://mlsanity.com/paper/2506.07460/) (2025) - [InterEdit: Navigating Text-Guided 3D Dyadic Human Motion Editing](https://mlsanity.com/paper/2603.13082/) (2026) - [Multi-Modal Controlled Coherent Motion Generation](https://mlsanity.com/paper/2609.11439/) (2026) - [Triangular Resampling for Long-Horizon Motion Generation](https://mlsanity.com/paper/2609.34697/) (2026) - [ELMA: Benchmarking Anaphoric Compositions for Long-Term Text-to-Motion Generation](https://mlsanity.com/paper/2610.13549/) (2026) ## Abstract (arXiv) In this paper, we address the challenging problem of long-term 3D human motion generation. Specifically, we aim to generate a long sequence of smoothly connected actions from a stream of multiple sentences (i.e., paragraph). Previous long-term motion generating approaches were mostly based on recurrent methods, using previously generated motion chunks as input for the next step. However, this approach has two drawbacks: 1) it relies on sequential datasets, which are expensive; 2) these methods yield unrealistic gaps between motions generated at each step. To address these issues, we introduce simple yet effective T2LM, a continuous long-term generation framework that can be trained without sequential data. T2LM comprises two components: a 1D-convolutional VQVAE, trained to compress motion to sequences of latent vectors, and a Transformer-based Text Encoder that predicts a latent sequence given an input text. At inference, a sequence of sentences is translated into a continuous stream of latent vectors. This is then decoded into a motion by the VQVAE decoder; the use of 1D convolutions with a local temporal receptive field avoids temporal inconsistencies between training and generated sequences. This simple constraint on the VQ-VAE allows it to be trained with short sequences only and produces smoother transitions. T2LM outperforms prior long-term generation models while overcoming the constraint of requiring sequential data; it is also competitive with SOTA single-action generation models.