mlsanityData to 7 Oct 2026

CoMo: Controllable Motion Generation through Language Guided Pose Code Editing

arXiv 2403.13900 ↗First posted 20 Mar 2024cs.CVcited by 66

Not on the map. Why?

Abstract (arXiv)

Text-to-motion models excel at efficient human motion generation, but existing approaches lack fine-grained controllability over the generation process. Consequently, modifying subtle postures within a motion or inserting new actions at specific moments remains a challenge, limiting the applicability of these methods in diverse scenarios. In light of these challenges, we introduce CoMo, a Controllable Motion generation model, adept at accurately generating and editing motions by leveraging the knowledge priors of large language models (LLMs). Specifically, CoMo decomposes motions into discrete and semantically meaningful pose codes, with each code encapsulating the semantics of a body part, representing elementary information such as "left knee slightly bent". Given textual inputs, CoMo autoregressively generates sequences of pose codes, which are then decoded into 3D motions. Leveraging pose codes as interpretable representations, an LLM can directly intervene in motion editing by adjusting the pose codes according to editing instructions. Experiments demonstrate that CoMo achieves competitive performance in motion generation compared to state-of-the-art models while, in human studies, CoMo substantially surpasses previous work in motion editing abilities.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 42 of its 51 references that are in the map. First posted on arXiv 20 Mar 2024.

Builds on

42 of 51 references.

References · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  6. 2022Human Motion Diffusion Modelcited by 1,062
  7. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  8. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
  9. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
  10. 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
  11. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  12. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 438
30 more
  1. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 406
  2. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 365
  3. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 357
  4. 2023Human Motion Diffusion as a Generative Priorcited by 354
  5. 2016The KIT Motion-Language Datasetcited by 334
  6. 2022EDGE: Editable Dance Generation From Musiccited by 284
  7. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 282
  8. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 237
  9. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 233
  10. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  11. 2022FLAME: Free-form Language-based Motion Synthesis & Editingcited by 217
  12. 2022Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memorycited by 210
  13. 2022Generating Holistic 3D Human Motion from Speechcited by 180
  14. 2022TEACH: Temporal Action Composition for 3D Humanscited by 173
  15. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
  16. 2022COUCH: Towards Controllable Human-Chair Interactionscited by 134
  17. 2023Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generationcited by 128
  18. 2023Synthesizing Diverse Human Motions in 3D Indoor Scenescited by 109
  19. 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editingcited by 105
  20. 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 93
  21. 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generationcited by 88
  22. 2022UDE: A Unified Driving Engine for Human Motion Generationcited by 67
  23. 2023SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generationcited by 62
  24. 2023C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characterscited by 53
  25. 2023Act As You Wish: Fine-Grained Control of Motion Diffusion Model with Hierarchical Semantic Graphscited by 44
  26. 2023Iterative Motion Editing with Natural Languagecited by 39
  27. 2023PoseFix: Correcting 3D Human Poses with Natural Languagecited by 38
  28. 2022Action-GPT: Leveraging Large-scale Language Models for Improved and Generalized Action Generationcited by 28
  29. 2023DiffMimic: Efficient Motion Mimicking with Differentiable Physicscited by 17
  30. 2023DiffusionPhase: Motion Diffusion in Frequency Domaincited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 27; 2026: 27.

66 papers.

Sort:

By area:

  1. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 78
  2. 2024MotionFix: Text-Driven 3D Human Motion Editingcited by 63
  3. 2024Motion-Agent: A Conversational Framework for Human Motion Generation with LLMscited by 46
  4. 2025SnapMoGen: Human Motion Generation from Expressive Textscited by 44
  5. 2024MaskControl: Spatio-Temporal Control for Masked Motion Synthesiscited by 37
  6. 2025SALAD: Skeleton-aware Latent Diffusion for Text-driven Motion Generation and Editingcited by 37
  7. 2024Unimotion: Unifying 3D Human Motion Synthesis and Understandingcited by 29
  8. 2025MG-MotionLLM: A Unified Framework for Motion Comprehension and Generation across Multiple Granularitiescited by 29
  9. 2025Absolute Coordinates Make Motion Generation Easycited by 27
  10. 2025SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Predictioncited by 26
  11. 2024Retrieving Semantics from the Deep: an RAG Solution for Gesture Synthesiscited by 25
  12. 2024KinMo: Kinematic-aware Human Motion Understanding and Generationcited by 21
38 more
  1. 2024CoMA: Compositional Human Motion Generation with Multi-modal Agentscited by 18
  2. 2024Pay Attention and Move Better: Harnessing Attention for Interactive Motion Generation and Training-free Editingcited by 17
  3. 2025SocialGen: Modeling Multi-Human Social Interaction with Language Modelscited by 14
  4. 2024SCENIC: Scene-aware Semantic Navigation with Instruction-guided Controlcited by 13
  5. 2025PRIMAL: Physically Reactive and Interactive Motor Model for Avatar Learningcited by 13
  6. 2025Dynamic Motion Blending for Versatile Motion Editingcited by 13
  7. 2025MOSPA: Human Motion Generation Driven by Spatial Audiocited by 11
  8. 2025Human-Centric Foundation Models: Perception, Generation and Agentic Modelingcited by 10
  9. 2025Text2Interact: High-Fidelity and Diverse Text-to-Two-Person Interaction Generationcited by 10
  10. 2025ReAlign: Text-to-Motion Generation via Step-Aware Reward-Guided Alignmentcited by 10
  11. 2024MotionWavelet: Human Motion Prediction via Wavelet Manifold Learningcited by 8
  12. 2025Motion Generation: A Survey of Generative Approaches and Benchmarkscited by 8
  13. 2025ModSkill: Physical Character Skill Modularizationcited by 7
  14. 2026FrankenMotion: Part-level Human Motion Generation and Compositioncited by 7
  15. 2024Lagrangian Motion Fields for Long-term Motion Generationcited by 5
  16. 2025MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editingcited by 5
  17. 2026Temporal Consistency-Aware Text-to-Motion Generationcited by 5
  18. 2025Mojito: LLM-Aided Motion Instructor with Jitter-Reduced Inertial Tokenscited by 4
  19. 2025FineXtrol: Controllable Motion Generation via Fine-Grained Textcited by 4
  20. 2026Omni-Supervised Motion Editing: Balancing Change and Invariance through Positive-Negative Learningcited by 4
  21. 2025MOST: Motion Diffusion Model for Rare Text via Temporal Clip Banzhaf Interactioncited by 3
  22. 2025SegMo: Segment-aligned Text to 3D Human Motion Generationcited by 3
  23. 2026EasyTune: Efficient Step-Aware Fine-Tuning for Diffusion-Based Motion Generationcited by 3
  24. 2025Dense Motion Captioningcited by 2
  25. 2025Pose-Guided Residual Refinement for Interpretable Text-to-Motion Generation and Editingcited by 2
  26. 2026Cross-Axis Feature Fusion with Joint-Wise Motion Difference Prediction for Text-Based 3D Human Motion Editingcited by 2
  27. 2025ACMo: Attribute Controllable Motion Generationcited by 1
  28. 2025How Much Do Large Language Models Know about Human Motion? A Case Study in 3D Avatar Controlcited by 1
  29. 2025SOSControl: Enhancing Human Motion Generation through Saliency-Aware Symbolic Orientation and Timing Controlcited by 1
  30. 2026LaMoGen: Language to Motion Generation Through LLM-Guided Symbolic Inferencecited by 1
  31. 2026ReactMotion: Generating Reactive Listener Motions from Speaker Utterancecited by 1
  32. 2026A Unified Conditional Flow for Motion Generation, Editing, and Intra-Structural Retargetingcited by 1
  33. 2026KV-Control: Parameter-Efficient K/V Injection for Trajectory-Controlled Text-to-Motioncited by 1
  34. 2026HumanCLAW: Can Vision-Language Models Act Through a Body?cited by 1
  35. 2026FineMoLA: Towards Fine-Grained Motion-Language Alignment from Clip-Level Supervisioncited by 1
  36. 2026Spatial Temporal Synergy: Balancing Change and Invariance in Text Driven 3D Human Motion Editingcited by 1
  37. 2025ReAlign: Bilingual Text-to-Motion Generation via Step-Aware Reward-Guided Alignmentcited by 0
  38. 2025Making Pose Representations More Expressive and Disentangled via Residual Vector Quantizationcited by 0

50 most cited of 66 All 66 (JSON)