# KinMo: Kinematic-aware Human Motion Understanding and Generation

arXiv: https://arxiv.org/abs/2411.15472 · HTML: https://mlsanity.com/paper/2411.15472/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 73 references that are in the map.
- First posted on arXiv 23 Nov 2024.

## Builds on (most cited first, 30 of 54)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020)
- [MotionCLIP: Exposing Human Motion Generation to CLIP Space](https://mlsanity.com/paper/2203.08063/) (2022)
- [MoMask: Generative Masked Modeling of 3D Human Motions](https://mlsanity.com/paper/2312.00063/) (2023)
- [TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts](https://mlsanity.com/paper/2207.01696/) (2022)
- [Human Motion Diffusion as a Generative Prior](https://mlsanity.com/paper/2303.01418/) (2023)
- [MACE: Mass Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2403.06135/) (2024)
- [Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance](https://mlsanity.com/paper/2403.14781/) (2024)
- [OmniControl: Control Any Joint at Any Time for Human Motion Generation](https://mlsanity.com/paper/2310.08580/) (2023)
- [TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis](https://mlsanity.com/paper/2305.00976/) (2023)
- [TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition](https://mlsanity.com/paper/2307.12493/) (2023)
- [MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model](https://mlsanity.com/paper/2404.19759/) (2024)
- [MMM: Generative Masked Motion Model](https://mlsanity.com/paper/2312.03596/) (2023)
- [Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advances](https://mlsanity.com/paper/2410.18775/) (2024)
- [FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editing](https://mlsanity.com/paper/2312.15004/) (2023)

## Built on by (most cited first, 21 of 21)

- [SemGes: Semantics-aware Co-Speech Gesture Generation using Semantic Coherence and Relevance Learning](https://mlsanity.com/paper/2507.19359/) (2025)
- [Caption Anything in Video: Fine-grained Object-centric Captioning via Spatiotemporal Multimodal Prompting](https://mlsanity.com/paper/2504.05541/) (2025)
- [FrankenMotion: Part-level Human Motion Generation and Composition](https://mlsanity.com/paper/2601.10909/) (2026)
- [Video-R4: Reinforcing Text-Rich Video Reasoning with Visual Rumination](https://mlsanity.com/paper/2511.17490/) (2025)
- [MMPerspective: Do MLLMs Understand Perspective? A Comprehensive Benchmark for Perspective Perception, Reasoning, and Robustness](https://mlsanity.com/paper/2505.20426/) (2025)
- [ActionPlan: Future-Aware Streaming Motion Synthesis via Frame-Level Action Planning](https://mlsanity.com/paper/2603.13500/) (2026)
- [SemConFlow: Semantic Grounding of Holistic Co-Speech Gesture Generation with Contrastive Flow-Matching](https://mlsanity.com/paper/2603.26553/) (2026)
- [SentiAvatar: Towards Expressive and Interactive Digital Humans](https://mlsanity.com/paper/2604.02908/) (2026)
- [Dense Motion Captioning](https://mlsanity.com/paper/2511.05369/) (2025)
- [MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation](https://mlsanity.com/paper/2512.13840/) (2025)
- [Fine-grained Motion Retrieval via Joint-Angle Motion Images and Token-Patch Late Interaction](https://mlsanity.com/paper/2603.09930/) (2026)
- [ReactMotion: Generating Reactive Listener Motions from Speaker Utterance](https://mlsanity.com/paper/2603.15083/) (2026)
- [Reconstruction-Anchored Diffusion Model for Text-to-Motion Generation](https://mlsanity.com/paper/2601.14788/) (2026)
- [InterEdit: Navigating Text-Guided 3D Dyadic Human Motion Editing](https://mlsanity.com/paper/2603.13082/) (2026)
- [LingoMotion: An Interpretable and Unambiguous Symbolic Representation for Human Motion](https://mlsanity.com/paper/2603.13578/) (2026)
- [TDMM-LM: Bridging Facial Understanding and Animation via Language Models](https://mlsanity.com/paper/2603.16936/) (2026)
- [Marrying Text-to-Motion Generation with Skeleton-Based Action Recognition](https://mlsanity.com/paper/2604.17090/) (2026)
- [MotionMERGE: A Multi-granular Framework for Human Motion Editing, Reasoning, Generation, and Explanation](https://mlsanity.com/paper/2605.18956/) (2026)
- [InteractGesture: Progressive Chunk Guidance for Continuous Streaming Co-Speech Gesture Control](https://mlsanity.com/paper/2608.25734/) (2026)
- [Multi-Modal Controlled Coherent Motion Generation](https://mlsanity.com/paper/2609.11439/) (2026)
- [BiMoGen: Bidirectional Motion-Text Generation via Unified Masked Discrete Diffusion](https://mlsanity.com/paper/2609.35407/) (2026)

## Abstract (arXiv)

Current human motion synthesis frameworks rely on global action descriptions, creating a modality gap that limits both motion understanding and generation capabilities. A single coarse description, such as run, fails to capture details such as variations in speed, limb positioning, and kinematic dynamics, leading to ambiguities between text and motion modalities. To address this challenge, we introduce KinMo, a unified framework built on a hierarchical describable motion representation that extends beyond global actions by incorporating kinematic group movements and their interactions. We design an automated annotation pipeline to generate high-quality, fine-grained descriptions for this decomposition, resulting in the KinMo dataset and offering a scalable and cost-efficient solution for dataset enrichment. To leverage these structured descriptions, we propose Hierarchical Text-Motion Alignment that progressively integrates additional motion details, thereby improving semantic motion understanding. Furthermore, we introduce a coarse-to-fine motion generation procedure to leverage enhanced spatial understanding to improve motion synthesis. Experimental results show that KinMo significantly improves motion understanding, demonstrated by enhanced text-motion retrieval performance and enabling more fine-grained motion generation and editing capabilities. Project Page: https://andypinxinliu.github.io/KinMo