mlsanityData to 14 Oct 2026

PlanMoGPT: Flow-Enhanced Progressive Planning for Text to Motion Synthesis

arXiv 2506.17912 ↗First posted 22 Jun 2025cs.CV cs.MMcited by 2

Not on the map. Why?

Abstract (arXiv)

Recent advances in large language models (LLMs) have enabled breakthroughs in many multimodal generation tasks, but a significant performance gap still exists in text-to-motion generation, where LLM-based methods lag far behind non-LLM methods. We identify the granularity of motion tokenization as a critical bottleneck: fine-grained tokenization induces local dependency issues, where LLMs overemphasize short-term coherence at the expense of global semantic alignment, while coarse-grained tokenization sacrifices motion details. To resolve this issue, we propose PlanMoGPT, an LLM-based framework integrating progressive planning and flow-enhanced fine-grained motion tokenization. First, our progressive planning mechanism leverages LLMs' autoregressive capabilities to hierarchically generate motion tokens by starting from sparse global plans and iteratively refining them into full sequences. Second, our flow-enhanced tokenizer doubles the downsampling resolution and expands the codebook size by eight times, minimizing detail loss during discretization, while a flow-enhanced decoder recovers motion nuances. Extensive experiments on text-to-motion benchmarks demonstrate that it achieves state-of-the-art performance, improving FID scores by 63.8% (from 0.380 to 0.141) on long-sequence generation while enhancing motion diversity by 49.9% compared to existing methods. The proposed framework successfully resolves the diversity-quality trade-off that plagues current non-LLM approaches, establishing new standards for text-to-motion generation.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 54 of its 62 references that are in the map. First posted on arXiv 22 Jun 2025.

Builds on

54 of 62 references.

References · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2023Visual Instruction Tuningcited by 7,869
  8. 2022Flow Matching for Generative Modelingcited by 5,702
  9. 2017Neural Discrete Representation Learningcited by 4,712
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  11. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  12. 2022Emergent Abilities of Large Language Modelscited by 2,512
38 more
  1. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  2. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  3. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  4. 2022Human Motion Diffusion Modelcited by 1,080
  5. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  6. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  7. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
  8. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  9. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  10. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  11. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
  12. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  13. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 408
  14. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
  15. 2023Human Motion Diffusion as a Generative Priorcited by 357
  16. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 287
  17. 2021BABEL: Bodies, Action and Behavior with English Labelscited by 265
  18. 2023Generating Images with Multimodal Language Modelscited by 260
  19. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  20. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  21. 2020Long-term Human Motion Prediction with Scene Contextcited by 192
  22. 2019Learning Individual Styles of Conversational Gesturecited by 191
  23. 2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 178
  24. 2023GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latentscited by 158
  25. 2021GOAL: Generating 4D Whole-Body Motion for Hand-Object Graspingcited by 144
  26. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  27. 2021Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoderscited by 109
  28. 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editingcited by 106
  29. 2023Human Motion Generation: A Surveycited by 96
  30. 2021SAGA: Stochastic Whole-Body Grasping with Contactcited by 91
  31. 2022ActFormer: A GAN-based Transformer towards General Action-Conditioned 3D Human Motion Generationcited by 77
  32. 2021Scene-aware Generative Network for Human Motion Synthesiscited by 70
  33. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 59
  34. 2024BAMM: Bidirectional Autoregressive Motion Modelcited by 55
  35. 2019Analyzing Input and Output Representations for Speech-Driven Gesture Generationcited by 52
  36. 2024Motion-Agent: A Conversational Framework for Human Motion Generation with LLMscited by 47
  37. 2023QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generationcited by 40
  38. 2023Motion Flow Matching for Human Motion Synthesis and Editingcited by 38

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.