mlsanityData to 14 Oct 2026

MotionGPT: Human Motion as a Foreign Language

arXiv 2306.14795 ↗First posted 26 Jun 2023cs.CV cs.CL cs.GRcited by 527

Not on the map. Why?

Cited by 78 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Though the advancement of pre-trained large language models unfolds, the exploration of building a unified model for language and other multi-modal data, such as motion, remains challenging and untouched so far. Fortunately, human motion displays a semantic coupling akin to human language, often perceived as a form of body language. By fusing language data with large-scale motion models, motion-language pre-training that can enhance the performance of motion-related tasks becomes feasible. Driven by this insight, we propose MotionGPT, a unified, versatile, and user-friendly motion-language model to handle multiple motion-relevant tasks. Specifically, we employ the discrete vector quantization for human motion and transfer 3D motion into motion tokens, similar to the generation process of word tokens. Building upon this "motion vocabulary", we perform language modeling on both motion and text in a unified manner, treating human motion as a specific language. Moreover, inspired by prompt learning, we pre-train MotionGPT with a mixture of motion-language data and fine-tune it on prompt-based question-and-answer tasks. Extensive experiments demonstrate that MotionGPT achieves state-of-the-art performances on multiple motion tasks including text-driven motion generation, motion captioning, motion prediction, and motion in-between.

Key facts. Cited by 527 papers in the map (data through 14 Oct 2026). Builds on 50 of its 73 references that are in the map. First posted on arXiv 26 Jun 2023.

Builds on

50 of 73 references.

References · 50

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2022Training language models to follow instructions with human feedbackcited by 15,971
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  9. 2020Denoising Diffusion Implicit Modelscited by 9,781
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  11. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  12. 2017Neural Discrete Representation Learningcited by 4,712
38 more
  1. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  5. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  7. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  8. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  9. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  10. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  11. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  12. 2022Human Motion Diffusion Modelcited by 1,080
  13. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  14. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
  15. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  16. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
  17. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  18. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 511
  19. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  20. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
  21. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  22. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 408
  23. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
  24. 2023Human Motion Diffusion as a Generative Priorcited by 357
  25. 2021AudioCLIP: Extending CLIP to Image, Text and Audiocited by 339
  26. 2016The KIT Motion-Language Datasetcited by 338
  27. 2022PromptSource: An Integrated Development Environment and Repository for Natural Language Promptscited by 289
  28. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  29. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  30. 2022FLAME: Free-form Language-based Motion Synthesis & Editingcited by 217
  31. 2022Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memorycited by 212
  32. 2020DLow: Diversifying Latent Flows for Diverse Human Motion Predictioncited by 185
  33. 2020We are More than Our Joints: Predicting how 3D Bodies Movecited by 115
  34. 2017Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networkscited by 61
  35. 2022BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Informationcited by 51
  36. 2022CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshescited by 41
  37. 2021PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imagingcited by 14
  38. 2017Sparse Photometric 3D Face Reconstruction Guided by Morphable Modelscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 37; 2024: 134; 2025: 183; 2026: 173.

527 papers.

Sort:

By area:

  1. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
  2. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
  3. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 238
  4. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planningcited by 210
  5. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 168
  6. 2023MMM: Generative Masked Motion Modelcited by 131
  7. 2023State of the Art on Diffusion Models for Visual Computingcited by 127
  8. 2023Universal Humanoid Motion Representations for Physics-Based Controlcited by 116
  9. 2024Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wildcited by 114
  10. 2024Motion Mamba: Efficient and Long Sequence Motion Generationcited by 112
  11. 2023HumanTOMATO: Text-aligned Whole-body Motion Generationcited by 104
  12. 2024MeshXL: Neural Coordinate Field for Generative 3D Foundation Modelscited by 103
38 more
  1. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 102
  2. 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 98
  3. 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
  4. 2024TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representationcited by 95
  5. 2023Unified Human-Scene Interaction via Prompted Chain-of-Contactscited by 94
  6. 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 94
  7. 2024MotionLLM: Understanding Human Behaviors from Human Motions and Videoscited by 90
  8. 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generationcited by 89
  9. 2024Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interactioncited by 86
  10. 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 86
  11. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 85
  12. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 84
  13. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  14. 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 80
  15. 2025Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generationcited by 77
  16. 2023ChatPose: Chatting about 3D Human Posecited by 72
  17. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 68
  18. 2024CoMo: Controllable Motion Generation through Language Guided Pose Code Editingcited by 67
  19. 2024OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitationcited by 67
  20. 2024Empowering LLMs to Understand and Generate Complex Vector Graphicscited by 66
  21. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  22. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  23. 2023AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyondcited by 63
  24. 2023MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representationscited by 60
  25. 2024LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioningcited by 59
  26. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 59
  27. 2024Taming Diffusion Probabilistic Models for Character Controlcited by 57
  28. 2024BAMM: Bidirectional Autoregressive Motion Modelcited by 55
  29. 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 53
  30. 2025KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skillscited by 52
  31. 2024StableMoFusion: Towards Robust and Efficient Diffusion-based Motion Generation Frameworkcited by 50
  32. 2024Generating Human Motion in 3D Scenes from Text Descriptionscited by 49
  33. 2024RoHM: Robust Human Motion Reconstruction via Diffusioncited by 48
  34. 2023OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllerscited by 47
  35. 2024Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompanimentcited by 47
  36. 2024Motion-Agent: A Conversational Framework for Human Motion Generation with LLMscited by 47
  37. 2024InterMask: 3D Human Interaction Generation via Collaborative Masked Modelingcited by 47
  38. 2025SnapMoGen: Human Motion Generation from Expressive Textscited by 47

50 most cited of 527 All 527 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023DiverseMotion: Towards Diverse Human Motion Generation via Discrete Diffusion
  2. 2023Fg-T2M: Fine-Grained Text-Driven Human Motion Generation via Diffusion Model
  3. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators
  4. 2024LGTM: Local-to-Global Text-Driven Human Motion Diffusion Model
  5. 2024A Unified Framework for Motion Reasoning and Generation in Human Interaction
  6. 2024VersatileMotion: A Unified Framework for Motion Synthesis and Comprehension
  7. 2026MoSAT: Human Motion Generation from Spatial Audio and Textual Description
  8. 2026HumanMoveVQA: Can Video MLLMs reason about human movement in videos?