Open-UniMo: Towards Unified Motion-Language Understanding and Generation in the Open World
Not on the map. Why?
Abstract (arXiv)
Unified motion generation and understanding is crucial for embodied AI systems that can both synthesize and interpret human actions in open-world environments. Existing motion-language models often treat motion as an auxiliary modality of a language model, leading to text-dominated representations and limited cross-modal interaction. Moreover, the next-token prediction paradigm is not naturally suited to long motion sequences, where autoregressive generation may accumulate prediction errors. To address these challenges, we propose Open-UniMo, a unified Large Motion-Language Model (LMLM) trained on million-scale open-world motion-language data. Open-UniMo promotes modality parity by extending Qwen's vocabulary of about 150K text tokens with 64K motion tokens, enabling motion and language to share a unified token space. We further introduce motion-consistent Chain-of-Thought reasoning as an intermediate representation to bridge language semantics and motion dynamics. Open-UniMo is trained with a two-stage pipeline, where supervised fine-tuning establishes CoT-guided bidirectional motion-language mapping and Group Relative Policy Optimization (GRPO) improves semantic alignment while mitigating cumulative errors in autoregressive motion-token generation. To support comprehensive evaluation, we propose Open-MoBench, a VLM-guided benchmark for assessing text-to-motion (T2M) generation, motion-to-text (M2T) understanding, and bidirectional consistency. Extensive experiments show that Open-UniMo achieves state-of-the-art performance on both conventional metrics and Open-MoBench. Furthermore, ablation studies reveal that M2T understanding is not primarily limited by motion-token vocabulary size; instead, coupling M2T with the learnable T2M generation path yields stronger cross-modal representations, demonstrating that generation can facilitate understanding in AR-based motion-language modeling.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 71 of its 81 references that are in the map. First posted on arXiv 13 Sep 2026.
Builds on
71 of 81 references.
References · 71
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
38 more
- 2022Human Motion Diffusion Modelcited by 1,080
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
- 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 688
- 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
- 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
- 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
- 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
- 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
- 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
- 2016The KIT Motion-Language Datasetcited by 338
- 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 272
- 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
- 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
- 2025Transfer between Modalities with MetaQueriescited by 236
- 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
- 2021Synthesis of Compositional Animations from Textual Descriptionscited by 178
- 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 166
- 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 146
- 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
- 2023DisCo: Disentangled Control for Realistic Human Dance Generationcited by 142
- 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 135
- 2025Emu3.5: Native Multimodal Models are World Learnerscited by 132
- 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 131
- 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editingcited by 106
- 2020Dance Revolution: Long-Term Dance Generation with Music via Curriculum Learningcited by 81
- 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 68
- 2024Liquid: Language Models are Scalable and Unified Multi-modal Generatorscited by 63
- 2017Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networkscited by 61
- 2025LHM: Large Animatable Human Reconstruction Model from a Single Image in Secondscited by 61
50 most cited of 71 All 71 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026UniWAM: Unified World-Action Model
- 2026MUGEN: A Unified Framework for Efficient Motion Understanding and Generation
- 2026LLaMo: Scaling Pretrained Language Models for Unified Motion Understanding and Generation with Continuous Autoregressive Tokens
- 2026LaMoGen: Language to Motion Generation Through LLM-Guided Symbolic Inference
- 2026Chain of World: World Model Thinking in Latent Motion
- 2026World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis
- 2024VersatileMotion: A Unified Framework for Motion Synthesis and Comprehension
- 2026Language Movement Primitives: Grounding Language Models in Robot Motion