mlsanityData to 7 Oct 2026

Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation

arXiv 2305.09662 ↗First posted 16 May 2023cs.CV cs.AIcited by 56

On the map: Diffusion models → Video generation

Abstract (arXiv)

Text-guided human motion generation has drawn significant interest because of its impactful applications spanning animation and robotics. Recently, application of diffusion models for motion generation has enabled improvements in the quality of generated motions. However, existing approaches are limited by their reliance on relatively small-scale motion capture data, leading to poor performance on more diverse, in-the-wild prompts. In this paper, we introduce Make-An-Animation, a text-conditioned human motion generation model which learns more diverse poses and prompts from large-scale image-text datasets, enabling significant improvement in performance over prior works. Make-An-Animation is trained in two stages. First, we train on a curated large-scale dataset of (text, static pseudo-pose) pairs extracted from image-text datasets. Second, we fine-tune on motion capture data, adding additional layers to model the temporal dimension. Unlike prior diffusion models for motion generation, Make-An-Animation uses a U-Net architecture similar to recent text-to-video generation models. Human evaluation of motion realism and alignment with input text shows that our model reaches state-of-the-art performance on text-to-motion generation.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 34 of its 41 references that are in the map. First posted on arXiv 16 May 2023.

Builds on

34 of 41 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Imagen Video: High Definition Video Generation with Diffusion Models5 Oct
  2. 2022Human Motion Diffusion Model29 Sep
  3. 2022FLAME: Free-form Language-based Motion Synthesis & Editing1 Sep
  4. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model31 Aug

Other references · 30

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  9. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  10. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  11. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  12. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
18 more
  1. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  2. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 479
  3. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 438
  4. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
  5. 2017On human motion prediction using recurrent neural networkscited by 336
  6. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  7. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 177
  8. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 152
  9. 2018Human Motion Prediction via Spatio-Temporal Inpaintingcited by 114
  10. 2021Playing for 3D Human Recoverycited by 63
  11. 2022Implicit Neural Representations for Variable Length Human Motion Generationcited by 61
  12. 2022MoDi: Unconditional Motion Synthesis from Diverse Datacited by 59
  13. 2021Style and Pose Control for Image Synthesis of Humans from a Single Monocular Viewcited by 53
  14. 2019Text Guided Person Image Synthesiscited by 22
  15. 2022TIPS: Text-Induced Pose Synthesiscited by 11
  16. 2020Adversarial Synthesis of Human Pose from Textcited by 7
  17. 2021Towards Better Adversarial Synthesis of Human Images from Textcited by 7
  18. 2023Text-Conditional Contextualized Avatars For Zero-Shot Personalizationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 22; 2025: 15; 2026: 9.

56 papers.

Sort:

By area:

  1. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 137
  2. 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
  3. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 89
  4. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 78
  5. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  6. 2023FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Modelscited by 55
  7. 2024Sora as a World Model? A Complete Survey on Text-to-Video Generationcited by 53
  8. 2024StableMoFusion: Towards Robust and Efficient Diffusion-based Motion Generation Frameworkcited by 48
  9. 2023OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllerscited by 47
  10. 2023MAS: Multi-view Ancestral Sampling for 3D motion generation using 2D diffusioncited by 37
  11. 2024Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generationcited by 35
  12. 2025HOIGPT: Learning Long Sequence Hand-Object Interaction with Language Modelscited by 32
38 more
  1. 2025Absolute Coordinates Make Motion Generation Easycited by 27
  2. 2024M2D2M: Multi-Motion Generation from Text with Discrete Diffusion Modelscited by 23
  3. 2023MotionScript: Natural Language Descriptions for Expressive 3D Human Motionscited by 21
  4. 2024Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motioncited by 16
  5. 2025AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion Modelscited by 15
  6. 2023Bridging the Gap between Human Motion and Action Semantics via Kinematic Phrasescited by 13
  7. 2024BAD: Bidirectional Auto-regressive Diffusion for Text-to-Motion Generationcited by 12
  8. 2024MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generationcited by 10
  9. 2025RL from Physical Feedback: Aligning Large Motion Models with Humanoid Controlcited by 10
  10. 2024Motion-2-To-3: Leveraging 2D Motion Data for 3D Motion Generationscited by 9
  11. 2024Exploring Text-to-Motion Generation with Human Preferencecited by 8
  12. 2024Textual Decomposition Then Sub-motion-space Scattering for Open-Vocabulary Motion Generationcited by 8
  13. 2025Text-driven Motion Generation: Overview, Challenges and Directionscited by 8
  14. 2025Motion Generation: A Survey of Generative Approaches and Benchmarkscited by 8
  15. 2023MCM: Multi-condition Motion Synthesis Framework for Multi-scenariocited by 7
  16. 2024AnimateMe: 4D Facial Expressions via Diffusion Modelscited by 6
  17. 2025Geometric Neural Distance Fields for Learning Human Motion Priorscited by 6
  18. 2025Interact2Ar: Full-Body Human-Human Interaction Generation via Autoregressive Diffusion Modelscited by 6
  19. 2024WheelPose: Data Synthesis Techniques to Improve Pose Estimation Performance on Wheelchair Userscited by 5
  20. 2024PianoMotion10M: Dataset and Benchmark for Hand Motion Generation in Piano Performancecited by 5
  21. 2023FG-MDM: Towards Zero-Shot Human Motion Generation via ChatGPT-Refined Descriptionscited by 4
  22. 2026ActionPlan: Future-Aware Streaming Motion Synthesis via Frame-Level Action Planningcited by 4
  23. 2024Shape Conditioned Human Motion Generation with Diffusion Modelcited by 3
  24. 2024Occluded Human Pose Estimation based on Limb Joint Augmentationcited by 3
  25. 2026EasyTune: Efficient Step-Aware Fine-Tuning for Diffusion-Based Motion Generationcited by 3
  26. 2024MCM: Multi-condition Motion Synthesis Frameworkcited by 2
  27. 2024Di2Pose: Discrete Diffusion Model for Occluded 3D Human Pose Estimationcited by 2
  28. 2024Text-to-Events: Synthetic Event Camera Streams from Conditional Text Inputcited by 2
  29. 2024Artificial Intelligence for Biomedical Video Generationcited by 2
  30. 2025MoCLIP: Motion-Aware Fine-Tuning and Distillation of CLIP for Human Motion Generationcited by 2
  31. 2025InterMamba: Efficient Human-Human Interaction Generation with Adaptive Spatio-Temporal Mambacited by 2
  32. 2026Language-Guided Transformer Tokenizer for Human Motion Generationcited by 2
  33. 2024Towards Practical Single-shot Motion Synthesiscited by 1
  34. 2026DancingBox: A Lightweight MoCap System for Character Animation from Physical Proxiescited by 1
  35. 2025EgoAnimate: Generating Human Animations from Egocentric top-down Viewscited by 0
  36. 2025LuKAN: A Kolmogorov-Arnold Network Framework for 3D Human Motion Predictioncited by 0
  37. 2025Virtual Fitting Room: Generating Arbitrarily Long Videos of Virtual Try-On from a Single Image -- Technical Previewcited by 0
  38. 2025LaMoGen: Laban Movement-Guided Diffusion for Text-to-Motion Generationcited by 0

50 most cited of 56 All 56 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Enhanced Fine-grained Motion Diffusion for Text-driven Human Motion Synthesis
  2. 2025Generating Human Motion Videos using a Cascaded Text-to-Video Framework
  3. 2025Strong and Controllable 3D Motion Generation
  4. 2024Towards motion from video diffusion models
  5. 2025HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generation
  6. 2024Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation
  7. 2023AMD: Autoregressive Motion Diffusion
  8. 2024GUESS:GradUally Enriching SyntheSis for Text-Driven Human Motion Generation