mlsanityData to 14 Oct 2026

A Plug-and-Play 2D Motion Interface for Real-World Motion Language Models

arXiv 2608.15984 ↗First posted 17 Aug 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Motion Language Models (MoLMs) typically understand human motions by tokenizing 3D motion and processing the resulting tokens using a language model. However, obtaining accurate 3D motions from monocular videos is challenging, limiting their real-world applicability. To address this issue, we introduce a plug-and-play 2D Motion Interface that enables 3D-pretrained MoLMs to accept 2D motion inputs without modifying or fine-tuning the original models. Experiments on public datasets show that our method achieves performance comparable to 3D motion inputs across multiple MoLMs and outperforms training MoLMs from scratch on 2D motions. We further construct a monocular real-world video motion evaluation dataset and introduce a real-video adapter, demonstrating the usefulness of 2D motions over 3D motions under the evaluated monocular pose-estimation setting. These results suggest that 2D motion provides a practical interface for deploying MoLMs in real-world motion understanding settings. Code is available at https://github.com/irajisamurai/2D-Motion-Interface.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 44 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

34 of 44 references.

References · 34

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2017Neural Discrete Representation Learningcited by 4,712
  5. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  6. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  7. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  8. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  9. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
  10. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 456
  11. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
  12. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
22 more
  1. 2016The KIT Motion-Language Datasetcited by 338
  2. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 272
  3. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 227
  4. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 206
  5. 2023WHAM: Reconstructing World-grounded Humans with Accurate 3D Motioncited by 206
  6. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
  7. 2022DiffPose: Toward More Reliable 3D Pose Estimationcited by 100
  8. 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 91
  9. 2023ChatPose: Chatting about 3D Human Posecited by 72
  10. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 59
  11. 2024M3GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generationcited by 38
  12. 2024LLMs are Good Action Recognizerscited by 37
  13. 2025MotionGPT3: Human Motion as a Second Modalitycited by 37
  14. 2025MG-MotionLLM: A Unified Framework for Motion Comprehension and Generation across Multiple Granularitiescited by 30
  15. 2024Towards Open Domain Text-Driven Synthesis of Multi-Person Motionscited by 29
  16. 2025SocialGen: Modeling Multi-Human Social Interaction with Language Modelscited by 14
  17. 2024Large Language Models are Zero-Shot Recognizers for Activities of Daily Livingcited by 13
  18. 2025FineMotion: A Dataset and Benchmark with both Spatial and Temporal Annotation for Fine-grained Motion Generation and Editingcited by 11
  19. 2023T2M-HiFiGPT: Generating High Quality Human Motion from Textual Descriptions with Residual Discrete Representationscited by 10
  20. 2025HiPART: Hierarchical Pose AutoRegressive Transformer for Occluded 3D Human Pose Estimationcited by 8
  21. 2024RapVerse: Coherent Vocals and Whole-Body Motions Generations from Textcited by 7
  22. 2024V-VIPE: Variational View Invariant Pose Embeddingcited by 3

Built on by

Not cited yet.