mlsanityData to 14 Oct 2026

MoMask: Generative Masked Modeling of 3D Human Motions

arXiv 2312.00063 ↗First posted 29 Nov 2023cs.CVcited by 400

Not on the map. Why?

Cited by 62 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce MoMask, a novel masked modeling framework for text-driven 3D human motion generation. In MoMask, a hierarchical quantization scheme is employed to represent human motion as multi-layer discrete motion tokens with high-fidelity details. Starting at the base layer, with a sequence of motion tokens obtained by vector quantization, the residual tokens of increasing orders are derived and stored at the subsequent layers of the hierarchy. This is consequently followed by two distinct bidirectional transformers. For the base-layer motion tokens, a Masked Transformer is designated to predict randomly masked motion tokens conditioned on text input at training stage. During generation (i.e. inference) stage, starting from an empty sequence, our Masked Transformer iteratively fills up the missing tokens; Subsequently, a Residual Transformer learns to progressively predict the next-layer tokens based on the results from current layer. Extensive experiments demonstrate that MoMask outperforms the state-of-art methods on the text-to-motion generation task, with an FID of 0.045 (vs e.g. 0.141 of T2M-GPT) on the HumanML3D dataset, and 0.228 (vs 0.514) on KIT-ML, respectively. MoMask can also be seamlessly applied in related tasks without further model fine-tuning, such as text-guided temporal inpainting.

Key facts. Cited by 400 papers in the map (data through 14 Oct 2026). Builds on 49 of its 55 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

49 of 55 references.

References · 49

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2022Classifier-Free Diffusion Guidancecited by 5,847
  5. 2017Neural Discrete Representation Learningcited by 4,712
  6. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  7. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  8. 2022Human Motion Diffusion Modelcited by 1,080
  9. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  10. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  11. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  12. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
37 more
  1. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  2. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  3. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
  4. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  5. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  6. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
  7. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  8. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 408
  9. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
  10. 2022MAGVIT: Masked Generative Video Transformercited by 346
  11. 2016The KIT Motion-Language Datasetcited by 338
  12. 2022EDGE: Editable Dance Generation From Musiccited by 287
  13. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 287
  14. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  15. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 246
  16. 2022MAGE: MAsked Generative Encoder to Unify Representation Learning and Image Synthesiscited by 227
  17. 2022FLAME: Free-form Language-based Motion Synthesis & Editingcited by 217
  18. 2022Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memorycited by 212
  19. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 178
  20. 2022TEACH: Temporal Action Composition for 3D Humanscited by 176
  21. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
  22. 2020Dance Revolution: Long-Term Dance Generation with Music via Curriculum Learningcited by 81
  23. 2022PoseGPT: Quantization-based 3D Human Motion Generation and Forecastingcited by 78
  24. 2023TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integrationcited by 76
  25. 2022UDE: A Unified Driving Engine for Human Motion Generationcited by 67
  26. 2023Priority-Centric Human Motion Generation in Discrete Latent Spacecited by 67
  27. 2017Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networkscited by 61
  28. 2022Implicit Neural Representations for Variable Length Human Motion Generationcited by 61
  29. 2017Deep Video Generation, Prediction and Completion of Human Action Sequencescited by 52
  30. 2019Learning Diverse Stochastic Human-Action Generators by Learning Smooth Latent Transitionscited by 46
  31. 2022Being Comes from Not-being: Open-vocabulary Text-to-Motion Generation with Wordless Trainingcited by 45
  32. 2023DiverseMotion: Towards Diverse Human Motion Generation via Discrete Diffusioncited by 37
  33. 2021Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Predictioncited by 31
  34. 2014Stacked Quantizers for Compositional Vector Compressioncited by 30
  35. 2017Performance Guaranteed Network Acceleration via High-Order Residual Quantizationcited by 26
  36. 2021Action2video: Generating Videos of Human 3D Actionscited by 15
  37. 2017Regularized Residual Quantization: a multi-layer sparse dictionary learning approachcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 57; 2025: 172; 2026: 171.

400 papers.

Sort:

By area:

  1. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 168
  2. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 102
  3. 2025SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Controlcited by 101
  4. 2024MotionLLM: Understanding Human Behaviors from Human Motions and Videoscited by 90
  5. 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 86
  6. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 85
  7. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 84
  8. 2025Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generationcited by 77
  9. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 68
  10. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  11. 2024MotionFix: Text-Driven 3D Human Motion Editingcited by 64
  12. 2024LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioningcited by 59
38 more
  1. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 59
  2. 2024BAMM: Bidirectional Autoregressive Motion Modelcited by 55
  3. 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 53
  4. 2025GENMO: A GENeralist Model for Human MOtioncited by 49
  5. 2024Motion-Agent: A Conversational Framework for Human Motion Generation with LLMscited by 47
  6. 2024InterMask: 3D Human Interaction Generation via Collaborative Masked Modelingcited by 47
  7. 2025SnapMoGen: Human Motion Generation from Expressive Textscited by 47
  8. 2024The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motioncited by 46
  9. 2025GestureLSM: Latent Shortcut based Co-Speech Gesture Generation with Spatial-Temporal Modelingcited by 44
  10. 2025Motion Anything: Any to Motion Generationcited by 42
  11. 2024MoGenTS: Motion Generation based on Spatial-Temporal Joint Modelingcited by 41
  12. 2025AnyTop: Character Animation Diffusion with Any Topologycited by 41
  13. 2025SALAD: Skeleton-aware Latent Diffusion for Text-driven Motion Generation and Editingcited by 39
  14. 2024M3GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generationcited by 38
  15. 2024MaskControl: Spatio-Temporal Control for Masked Motion Synthesiscited by 38
  16. 2024SMooDi: Stylized Motion Diffusion Modelcited by 37
  17. 2025MotionGPT3: Human Motion as a Second Modalitycited by 37
  18. 2026Kimodo: Scaling Controllable Human Motion Generationcited by 37
  19. 2025HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generationcited by 36
  20. 2024SemTalk: Holistic Co-speech Motion Generation with Frame-level Semantic Emphasiscited by 32
  21. 2024MotionChain: Conversational Motion Controllers via Multimodal Promptscited by 30
  22. 2025MotionLab: Unified Human Motion Generation and Editing via the Motion-Condition-Motion Paradigmcited by 30
  23. 2025MG-MotionLLM: A Unified Framework for Motion Comprehension and Generation across Multiple Granularitiescited by 30
  24. 2024in2IN: Leveraging individual Information to Generate Human INteractionscited by 29
  25. 2025Absolute Coordinates Make Motion Generation Easycited by 29
  26. 2024TIMotion: Temporal and Interactive Framework for Efficient Human-Human Motion Generationcited by 27
  27. 2025SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Predictioncited by 27
  28. 2025A Survey on Human Interaction Motion Generationcited by 25
  29. 2024InfiniMotion: Mamba Boosts Memory in Transformer for Arbitrary Long Motion Generationcited by 23
  30. 2024Light-T2M: A Lightweight and Fast Model for Text-to-motion Generationcited by 23
  31. 2024Scaling Large Motion Models with Million-Level Human Motionscited by 22
  32. 2025The Quest for Generalizable Motion Generation: Data, Model, and Evaluationcited by 22
  33. 2024KinMo: Kinematic-aware Human Motion Understanding and Generationcited by 21
  34. 2025DuetGen: Music Driven Two-Person Dance Generation via Hierarchical Masked Modelingcited by 21
  35. 2024SATO: Stable Text-to-Motion Frameworkcited by 20
  36. 2025ChainHOI: Joint-based Kinematic Chain Modeling for Human-Object Interaction Generationcited by 20
  37. 2024Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motioncited by 19
  38. 2024DisCoRD: Discrete Tokens to Continuous Motion via Rectified Flow Decodingcited by 19

50 most cited of 400 All 400 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023MMM: Generative Masked Motion Model
  2. 2024Mogo: RQ Hierarchical Causal Transformer for High-Quality 3D Human Motion Generation
  3. 2025Strong and Controllable 3D Motion Generation
  4. 2023Make-An-Animation: Large-Scale Text-conditional 3D Human Motion Generation
  5. 2024Dynamic Motion Synthesis: Masked Audio-Text Conditioned Spatio-Temporal Transformers
  6. 2023T2M-HiFiGPT: Generating High Quality Human Motion from Textual Descriptions with Residual Discrete Representations
  7. 2022Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Model
  8. 2025UniMo: Unifying 2D Video and 3D Human Motion with an Autoregressive Framework