PlanMoGPT: Flow-Enhanced Progressive Planning for Text to Motion Synthesis
Not on the map. Why?
Abstract (arXiv)
Recent advances in large language models (LLMs) have enabled breakthroughs in many multimodal generation tasks, but a significant performance gap still exists in text-to-motion generation, where LLM-based methods lag far behind non-LLM methods. We identify the granularity of motion tokenization as a critical bottleneck: fine-grained tokenization induces local dependency issues, where LLMs overemphasize short-term coherence at the expense of global semantic alignment, while coarse-grained tokenization sacrifices motion details. To resolve this issue, we propose PlanMoGPT, an LLM-based framework integrating progressive planning and flow-enhanced fine-grained motion tokenization. First, our progressive planning mechanism leverages LLMs' autoregressive capabilities to hierarchically generate motion tokens by starting from sparse global plans and iteratively refining them into full sequences. Second, our flow-enhanced tokenizer doubles the downsampling resolution and expands the codebook size by eight times, minimizing detail loss during discretization, while a flow-enhanced decoder recovers motion nuances. Extensive experiments on text-to-motion benchmarks demonstrate that it achieves state-of-the-art performance, improving FID scores by 63.8% (from 0.380 to 0.141) on long-sequence generation while enhancing motion diversity by 49.9% compared to existing methods. The proposed framework successfully resolves the diversity-quality trade-off that plagues current non-LLM approaches, establishing new standards for text-to-motion generation.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 54 of its 62 references that are in the map. First posted on arXiv 22 Jun 2025.
Builds on
54 of 62 references.
References · 54
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2023GPT-4 Technical Reportcited by 18,396
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
- 2023Visual Instruction Tuningcited by 7,869
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2017Neural Discrete Representation Learningcited by 4,712
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
- 2022Emergent Abilities of Large Language Modelscited by 2,512
38 more
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
- 2022Human Motion Diffusion Modelcited by 1,080
- 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
- 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
- 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
- 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
- 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
- 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
- 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
- 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
- 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 408
- 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
- 2023Human Motion Diffusion as a Generative Priorcited by 357
- 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 287
- 2021BABEL: Bodies, Action and Behavior with English Labelscited by 265
- 2023Generating Images with Multimodal Language Modelscited by 260
- 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
- 2021Stochastic Scene-Aware Motion Predictioncited by 203
- 2020Long-term Human Motion Prediction with Scene Contextcited by 192
- 2019Learning Individual Styles of Conversational Gesturecited by 191
- 2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 178
- 2023GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latentscited by 158
- 2021GOAL: Generating 4D Whole-Body Motion for Hand-Object Graspingcited by 144
- 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
- 2021Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoderscited by 109
- 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editingcited by 106
- 2023Human Motion Generation: A Surveycited by 96
- 2021SAGA: Stochastic Whole-Body Grasping with Contactcited by 91
- 2022ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generationcited by 77
- 2021Scene-aware Generative Network for Human Motion Synthesiscited by 70
- 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 59
- 2024BAMM: Bidirectional Autoregressive Motion Modelcited by 55
- 2019Analyzing Input and Output Representations for Speech-Driven Gesture Generationcited by 52
- 2024Motion-Agent: A Conversational Framework for Human Motion Generation with LLMscited by 47
- 2023QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generationcited by 40
- 2023Motion Flow Matching for Human Motion Synthesis and Editingcited by 38
50 most cited of 54 All 54 (JSON)
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.