MotionMERGE: A Multi-granular Framework for Human Motion Editing, Reasoning, Generation, and Explanation
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Recent motion-language models unify tasks like comprehension and generation but operate at a coarse granularity, lacking fine-grained understanding and nuanced control over body parts needed for animation or interaction. This stems from fundamental issues in both the model and the data, in which the model can't focus on motion's localized pattern, and the training data lacks fine-grained supervision. To tackle this, we propose MotionMERGE, a unified framework that bridges the granularity gap. First, we pioneer the study of fine-grained languageguided motion control, including detailed understanding and localized editing, by explicitly modeling motion at part and temporal levels within a single LLM, thereby endowing the model with robust priors for precise control. Second, we design ReasoningAware Granularity-Synergy pre-training, a novel strategy that employs joint supervision for cross-granularity alignment, temporal grounding, localized alignment, motion coherency, and motion-grounded chain-of-thought (CoT) reasoning. This equips the model with fine-grained motion-language alignment, crossgranularity synergy, and explicit reasoning ability. Third, we curate MotionFineEdit, a large-scale dataset (837K atomic + 144K complex triplets) with the first fine-grained spatio-temporal corrective instructions and motion-grounded CoT annotations, establishing a new benchmark for fine-grained text-driven motion editing and motion-grounded reasoning. Extensive experiments demonstrate the capability of MotionMERGE for more precise motion generation, understanding, and editing, and compelling zero-shot generalization to other complex motion tasks. This work represents a significant step toward models that interact with motion in finer granularity and human-like reasoning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 85 of its 97 references that are in the map. First posted on arXiv 18 May 2026.
Builds on
85 of 97 references.
Closest ancestors · 4
Same direction, newest first
- 2025EvolveNav: Empowering LLM-Based Vision-Language Navigation via Self-Improving Embodied Reasoning2 Jun
- 2024A Unified Framework for Motion Reasoning and Generation in Human Interaction8 Oct
- 2023Motion Question Answering via Modular Motion Programs15 May
- 2022Toward 3D Spatial Reasoning for Human-like Text-based Visual Question Answering21 Sep
Other references · 81
- 2023GPT-4 Technical Reportcited by 18,396
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2023Visual Instruction Tuningcited by 7,869
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2017Neural Discrete Representation Learningcited by 4,712
- 2024GPT-4o System Cardcited by 4,588
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
- 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
- 2022Human Motion Diffusion Modelcited by 1,080
38 more
- 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
- 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 615
- 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
- 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
- 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
- 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
- 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
- 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
- 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 408
- 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
- 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
- 2016The KIT Motion-Language Datasetcited by 338
- 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
- 2022EDGE: Editable Dance Generation From Musiccited by 287
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
- 2023Pengi: An Audio Language Model for Audio Taskscited by 220
- 2022FLAME: Free-form Language-based Motion Synthesis & Editingcited by 217
- 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
- 2021DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformercited by 111
- 2024CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character controlcited by 109
- 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editingcited by 106
- 2023Human Motion Generation: A Surveycited by 96
- 2020Unpaired Motion Style Transfer from Video to Animationcited by 94
- 2024Multi-Track Timeline Control for Text-Driven 3D Human Motion Generationcited by 91
- 2024MotionLLM: Understanding Human Behaviors from Human Motions and Videoscited by 90
- 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 86
- 2024From Audio to Photoreal Embodiment: Synthesizing Humans in Conversationscited by 82
- 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 80
- 2023Fg-T2M: Fine-Grained Text-Driven Human Motion Generation via Diffusion Modelcited by 79
- 2024CoMo: Controllable Motion Generation through Language Guided Pose Code Editingcited by 67
- 2022Motion Puzzle: Arbitrary Motion Style Transfer by Body Partcited by 64
- 2024MotionFix: Text-Driven 3D Human Motion Editingcited by 64
- 2023AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyondcited by 63
- 2022MoDi: Unconditional Motion Synthesis from Diverse Datacited by 59
- 2024LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioningcited by 59
- 2024ParCo: Part-Coordinating Text-to-Motion Synthesiscited by 55
50 most cited of 81 All 81 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editing
- 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understanding
- 2026MUGEN: A Unified Framework for Efficient Motion Understanding and Generation
- 2026Motion-as-Prompt: Enhancing Motion Reasoning in Multimodal Large Language Models via Motion-Guided Cross-Frame Visual Prompting
- 2023Act As You Wish: Fine-Grained Control of Motion Diffusion Model with Hierarchical Semantic Graphs
- 2025MotionGPT3: Human Motion as a Second Modality
- 2026UniMo: Unified Motion Generation and Understanding with Chain of Thought
- 2026MoRL: Reinforced Reasoning for Unified Motion Understanding and Generation