# MG-MotionLLM: A Unified Framework for Motion Comprehension and Generation across Multiple Granularities

arXiv: https://arxiv.org/abs/2504.02478 · HTML: https://mlsanity.com/paper/2504.02478/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 35 references that are in the map.
- First posted on arXiv 3 Apr 2025.

## Builds on (most cited first, 29 of 29)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020)
- [MotionCLIP: Exposing Human Motion Generation to CLIP Space](https://mlsanity.com/paper/2203.08063/) (2022)
- [MoMask: Generative Masked Modeling of 3D Human Motions](https://mlsanity.com/paper/2312.00063/) (2023)
- [TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts](https://mlsanity.com/paper/2207.01696/) (2022)
- [EDGE: Editable Dance Generation From Music](https://mlsanity.com/paper/2211.10658/) (2022)
- [FLAME: Free-form Language-based Motion Synthesis & Editing](https://mlsanity.com/paper/2209.00349/) (2022)
- [MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators](https://mlsanity.com/paper/2306.10900/) (2023)
- [FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editing](https://mlsanity.com/paper/2312.15004/) (2023)
- [Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation](https://mlsanity.com/paper/2401.08559/) (2024)
- [Seamless Human Motion Composition with Blended Positional Encodings](https://mlsanity.com/paper/2402.15509/) (2024)
- [CoMo: Controllable Motion Generation through Language Guided Pose Code Editing](https://mlsanity.com/paper/2403.13900/) (2024)
- [M3GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generation](https://mlsanity.com/paper/2405.16273/) (2024)
- [Action-GPT: Leveraging Large-scale Language Models for Improved and Generalized Action Generation](https://mlsanity.com/paper/2211.15603/) (2022)
- [MotionScript: Natural Language Descriptions for Expressive 3D Human Motions](https://mlsanity.com/paper/2312.12634/) (2023)
- [SemanticBoost: Elevating Motion Generation with Augmented Textual Cues](https://mlsanity.com/paper/2310.20323/) (2023)
- [Dynamic data sampler for cross-language transfer learning in large language models](https://mlsanity.com/paper/2405.10626/) (2024)

## Built on by (most cited first, 30 of 30)

- [MotionGPT3: Human Motion as a Second Modality](https://mlsanity.com/paper/2506.24086/) (2025)
- [ReAlign: Text-to-Motion Generation via Step-Aware Reward-Guided Alignment](https://mlsanity.com/paper/2511.19217/) (2025)
- [IRG-MotionLLM: Interleaving Motion Generation, Assessment and Refinement for Text-to-Motion Generation](https://mlsanity.com/paper/2512.10730/) (2025)
- [MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editing](https://mlsanity.com/paper/2509.23635/) (2025)
- [Temporal Consistency-Aware Text-to-Motion Generation](https://mlsanity.com/paper/2602.18057/) (2026)
- [X-MoGen: Unified Motion Generation across Humans and Animals](https://mlsanity.com/paper/2508.05162/) (2025)
- [Bridging Semantic and Kinematic Conditions with Diffusion-based Discrete Motion Tokenizer](https://mlsanity.com/paper/2603.19227/) (2026)
- [UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation](https://mlsanity.com/paper/2603.22282/) (2026)
- [SentiAvatar: Towards Expressive and Interactive Digital Humans](https://mlsanity.com/paper/2604.02908/) (2026)
- [U-Mind: A Unified Framework for Real-Time Multimodal Interaction with Audiovisual Generation](https://mlsanity.com/paper/2602.23739/) (2026)
- [MotionVLA: Vision-Language-Action Model for Humanoid Motion](https://mlsanity.com/paper/2606.15142/) (2026)
- [SIGNER: Temporally Grounded Sign Language Generation via Time-Resolved Conditioning](https://mlsanity.com/paper/2506.07460/) (2025)
- [OmniMoGen: Unifying Human Motion Generation via Learning from Interleaved Text-Motion Instructions](https://mlsanity.com/paper/2512.19159/) (2025)
- [ReactMotion: Generating Reactive Listener Motions from Speaker Utterance](https://mlsanity.com/paper/2603.15083/) (2026)
- [FineMoLA: Towards Fine-Grained Motion-Language Alignment from Clip-Level Supervision](https://mlsanity.com/paper/2608.01392/) (2026)
- [PEMUTA: Pedagogically-Enriched Multi-Granular Undergraduate Thesis Assessment](https://mlsanity.com/paper/2507.19556/) (2025)
- [MoGIC: Boosting Motion Generation via Intention Understanding and Visual Context](https://mlsanity.com/paper/2510.02722/) (2025)
- [GeoMotionGPT: Geometry-Aligned Motion Understanding with Large Language Models](https://mlsanity.com/paper/2601.07632/) (2026)
- [Predictive Reasoning with Augmented Anomaly Contrastive Learning for Compositional Visual Relations](https://mlsanity.com/paper/2603.01125/) (2026)
- [Marrying Text-to-Motion Generation with Skeleton-Based Action Recognition](https://mlsanity.com/paper/2604.17090/) (2026)
- [Encoder-Free Human Motion Understanding via Structured Motion Descriptions](https://mlsanity.com/paper/2604.21668/) (2026)
- [MotionMERGE: A Multi-granular Framework for Human Motion Editing, Reasoning, Generation, and Explanation](https://mlsanity.com/paper/2605.18956/) (2026)
- [MUGEN: A Unified Framework for Efficient Motion Understanding and Generation](https://mlsanity.com/paper/2607.27581/) (2026)
- [Bridging the Gap Between Semantics and Reconstruction:Unifying Sign Language Translation and Production](https://mlsanity.com/paper/2608.09045/) (2026)
- [A Plug-and-Play 2D Motion Interface for Real-World Motion Language Models](https://mlsanity.com/paper/2608.15984/) (2026)
- [Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction Analysis](https://mlsanity.com/paper/2608.20312/) (2026)
- [Open-UniMo: Towards Unified Motion-Language Understanding and Generation in the Open World](https://mlsanity.com/paper/2609.14615/) (2026)
- [Can Motion-Language Models Ground Structure? STRIDE for Evaluating the Evaluators](https://mlsanity.com/paper/2609.32462/) (2026)
- [BiMoGen: Bidirectional Motion-Text Generation via Unified Masked Discrete Diffusion](https://mlsanity.com/paper/2609.35407/) (2026)
- [Parasitic Co-Denoising: Unlocking 3D Human Motion Generation in a Frozen Video Diffusion Model](https://mlsanity.com/paper/2610.03047/) (2026)

## Abstract (arXiv)

Recent motion-aware large language models have demonstrated promising potential in unifying motion comprehension and generation. However, existing approaches primarily focus on coarse-grained motion-text modeling, where text describes the overall semantics of an entire motion sequence in just a few words. This limits their ability to handle fine-grained motion-relevant tasks, such as understanding and controlling the movements of specific body parts. To overcome this limitation, we pioneer MG-MotionLLM, a unified motion-language model for multi-granular motion comprehension and generation. We further introduce a comprehensive multi-granularity training scheme by incorporating a set of novel auxiliary tasks, such as localizing temporal boundaries of motion segments via detailed text as well as motion detailed captioning, to facilitate mutual reinforcement for motion-text modeling across various levels of granularity. Extensive experiments show that our MG-MotionLLM achieves superior performance on classical text-to-motion and motion-to-text tasks, and exhibits potential in novel fine-grained motion comprehension and editing tasks. Project page: CVI-SZU/MG-MotionLLM