mlsanityData to 14 Oct 2026

T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations

arXiv 2301.06052 ↗First posted 15 Jan 2023cs.CVcited by 588

Not on the map. Why?

Cited by 101 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this work, we investigate a simple and must-known conditional generative framework based on Vector Quantised-Variational AutoEncoder (VQ-VAE) and Generative Pre-trained Transformer (GPT) for human motion generation from textural descriptions. We show that a simple CNN-based VQ-VAE with commonly used training recipes (EMA and Code Reset) allows us to obtain high-quality discrete representations. For GPT, we incorporate a simple corruption strategy during the training to alleviate training-testing discrepancy. Despite its simplicity, our T2M-GPT shows better performance than competitive approaches, including recent diffusion-based approaches. For example, on HumanML3D, which is currently the largest dataset, we achieve comparable performance on the consistency between text and generated motion (R-Precision), but with FID 0.116 largely outperforming MotionDiffuse of 0.630. Additionally, we conduct analyses on HumanML3D and observe that the dataset size is a limitation of our approach. Our work suggests that VQ-VAE still remains a competitive approach for human motion generation.

Key facts. Cited by 588 papers in the map (data through 14 Oct 2026). Builds on 53 of its 76 references that are in the map. First posted on arXiv 15 Jan 2023.

Builds on

53 of 76 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2017Neural Discrete Representation Learningcited by 4,712
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  10. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  11. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  12. 2022Human Motion Diffusion Modelcited by 1,080
38 more
  1. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  2. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
  3. 2020Jukebox: A Generative Model for Musiccited by 542
  4. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 524
  5. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  6. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
  7. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
  8. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  9. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 408
  10. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
  11. 2019SinGAN: Learning a Generative Model from a Single Natural Imagecited by 348
  12. 2016The KIT Motion-Language Datasetcited by 338
  13. 2017On human motion prediction using recurrent neural networkscited by 338
  14. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  15. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  16. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 246
  17. 2021Robust Motion In-betweeningcited by 229
  18. 2022Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memorycited by 212
  19. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 178
  20. 2017HP-GAN: Probabilistic 3D human motion prediction via GANcited by 176
  21. 2022TEACH: Temporal Action Composition for 3D Humanscited by 176
  22. 2020History Repeats Itself: Human Motion Prediction via Motion Attentioncited by 165
  23. 2018Human Motion Prediction via Spatio-Temporal Inpaintingcited by 114
  24. 2018QuaterNet: A Quaternion-based Recurrent Model for Human Motioncited by 109
  25. 2017Text2Action: Generative Adversarial Synthesis from Language to Actioncited by 106
  26. 2021Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agentscited by 106
  27. 2022Back to MLP: A Simple Baseline for Human Motion Predictioncited by 100
  28. 2017Deep representation learning for human motion prediction and classificationcited by 98
  29. 2018MT-VAE: Learning Motion Transformations to Generate Multimodal Human Dynamicscited by 86
  30. 2020Learning to Generate Diverse Dance Motions with Transformercited by 85
  31. 2022PoseGPT: Quantization-based 3D Human Motion Generation and Forecastingcited by 78
  32. 2020Convolutional Autoencoders for Human Motion Infillingcited by 73
  33. 2018The challenge of realistic music generation: modelling raw audio at scalecited by 63
  34. 2021Single-Shot Motion Completion with Transformercited by 59
  35. 2020Hierarchical Quantized Autoencoderscited by 57
  36. 2021Rhythm is a Dancer: Music-Driven Motion Synthesis with Global Structurecited by 53
  37. 2018Recurrent Transition Networks for Character Locomotioncited by 52
  38. 2022MotionMixer: MLP-based 3D Human Body Pose Forecastingcited by 42

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 63; 2024: 146; 2025: 192; 2026: 187.

588 papers.

Sort:

By area:

  1. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  2. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
  3. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 287
  4. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 238
  5. 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 228
  6. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 227
  7. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 206
  8. 2023TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesiscited by 198
  9. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  10. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 168
  11. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
  12. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 138
38 more
  1. 2024Scaling Up Dynamic Human-Scene Interaction Modelingcited by 137
  2. 2023MMM: Generative Masked Motion Modelcited by 131
  3. 2023AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanismcited by 117
  4. 2024Large Language Models for Time Series: A Surveycited by 114
  5. 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editingcited by 106
  6. 2023HumanTOMATO: Text-aligned Whole-body Motion Generationcited by 104
  7. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 102
  8. 2025SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Controlcited by 101
  9. 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 98
  10. 2023Human Motion Generation: A Surveycited by 96
  11. 2024TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representationcited by 95
  12. 2023Unified Human-Scene Interaction via Prompted Chain-of-Contactscited by 94
  13. 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 94
  14. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 93
  15. 2024MotionLLM: Understanding Human Behaviors from Human Motions and Videoscited by 90
  16. 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generationcited by 89
  17. 2024Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interactioncited by 86
  18. 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 86
  19. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 85
  20. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 84
  21. 2023HumanMAC: Masked Motion Completion for Human Motion Predictioncited by 81
  22. 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 80
  23. 2024Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordancecited by 80
  24. 2023Fg-T2M: Fine-Grained Text-Driven Human Motion Generation via Diffusion Modelcited by 79
  25. 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 76
  26. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 68
  27. 2023Priority-Centric Human Motion Generation in Discrete Latent Spacecited by 67
  28. 2024CoMo: Controllable Motion Generation through Language Guided Pose Code Editingcited by 67
  29. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  30. 2023SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generationcited by 64
  31. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  32. 2023AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyondcited by 63
  33. 2023ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactionscited by 62
  34. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 61
  35. 2023MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representationscited by 60
  36. 2024LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioningcited by 59
  37. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 59
  38. 2023FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Modelscited by 55

50 most cited of 588 All 588 (JSON)