mlsanityData to 14 Oct 2026

Open-UniMo: Towards Unified Motion-Language Understanding and Generation in the Open World

arXiv 2609.14615 ↗First posted 13 Sep 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Unified motion generation and understanding is crucial for embodied AI systems that can both synthesize and interpret human actions in open-world environments. Existing motion-language models often treat motion as an auxiliary modality of a language model, leading to text-dominated representations and limited cross-modal interaction. Moreover, the next-token prediction paradigm is not naturally suited to long motion sequences, where autoregressive generation may accumulate prediction errors. To address these challenges, we propose Open-UniMo, a unified Large Motion-Language Model (LMLM) trained on million-scale open-world motion-language data. Open-UniMo promotes modality parity by extending Qwen's vocabulary of about 150K text tokens with 64K motion tokens, enabling motion and language to share a unified token space. We further introduce motion-consistent Chain-of-Thought reasoning as an intermediate representation to bridge language semantics and motion dynamics. Open-UniMo is trained with a two-stage pipeline, where supervised fine-tuning establishes CoT-guided bidirectional motion-language mapping and Group Relative Policy Optimization (GRPO) improves semantic alignment while mitigating cumulative errors in autoregressive motion-token generation. To support comprehensive evaluation, we propose Open-MoBench, a VLM-guided benchmark for assessing text-to-motion (T2M) generation, motion-to-text (M2T) understanding, and bidirectional consistency. Extensive experiments show that Open-UniMo achieves state-of-the-art performance on both conventional metrics and Open-MoBench. Furthermore, ablation studies reveal that M2T understanding is not primarily limited by motion-token vocabulary size; instead, coupling M2T with the learnable T2M generation path yields stronger cross-modal representations, demonstrating that generation can facilitate understanding in AR-based motion-language modeling.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 71 of its 81 references that are in the map. First posted on arXiv 13 Sep 2026.

Builds on

71 of 81 references.

References · 71

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  8. 2024Qwen2.5 Technical Reportcited by 4,942
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  10. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  12. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
38 more
  1. 2022Human Motion Diffusion Modelcited by 1,080
  2. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  3. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  4. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  5. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  6. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  7. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  8. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  9. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  10. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  11. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 688
  12. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  13. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
  14. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  15. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  16. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
  17. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  18. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
  19. 2016The KIT Motion-Language Datasetcited by 338
  20. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 272
  21. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  22. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
  23. 2025Transfer between Modalities with MetaQueriescited by 236
  24. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
  25. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 178
  26. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 166
  27. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 146
  28. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
  29. 2023DisCo: Disentangled Control for Realistic Human Dance Generationcited by 142
  30. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 135
  31. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 132
  32. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 131
  33. 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editingcited by 106
  34. 2020Dance Revolution: Long-Term Dance Generation with Music via Curriculum Learningcited by 81
  35. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 68
  36. 2024Liquid: Language Models are Scalable and Unified Multi-modal Generatorscited by 63
  37. 2017Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networkscited by 61
  38. 2025LHM: Large Animatable Human Reconstruction Model from a Single Image in Secondscited by 61

50 most cited of 71 All 71 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026UniWAM: Unified World-Action Model
  2. 2026MUGEN: A Unified Framework for Efficient Motion Understanding and Generation
  3. 2026LLaMo: Scaling Pretrained Language Models for Unified Motion Understanding and Generation with Continuous Autoregressive Tokens
  4. 2026LaMoGen: Language to Motion Generation Through LLM-Guided Symbolic Inference
  5. 2026Chain of World: World Model Thinking in Latent Motion
  6. 2026World-Language-Action Model for Unified World Modeling, Language Reasoning, and Action Synthesis
  7. 2024VersatileMotion: A Unified Framework for Motion Synthesis and Comprehension
  8. 2026Language Movement Primitives: Grounding Language Models in Robot Motion