# How to Move Your Dragon: Text-to-Motion Synthesis for Large-Vocabulary Objects

arXiv: https://arxiv.org/abs/2503.04257 · HTML: https://mlsanity.com/paper/2503.04257/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 6 Mar 2025.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)
- [MoMask: Generative Masked Modeling of 3D Human Motions](https://mlsanity.com/paper/2312.00063/) (2023)
- [The KIT Motion-Language Dataset](https://mlsanity.com/paper/1607.03827/) (2016)
- [StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation](https://mlsanity.com/paper/2405.01434/) (2024)
- [DreamGaussian4D: Generative 4D Gaussian Splatting](https://mlsanity.com/paper/2312.17142/) (2023)
- [L4GM: Large 4D Gaussian Reconstruction Model](https://mlsanity.com/paper/2406.10324/) (2024)
- [FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editing](https://mlsanity.com/paper/2312.15004/) (2023)
- [Single Motion Diffusion](https://mlsanity.com/paper/2302.05905/) (2023)
- [OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllers](https://mlsanity.com/paper/2312.08985/) (2023)
- [MAS: Multi-view Ancestral Sampling for 3D motion generation using 2D diffusion](https://mlsanity.com/paper/2310.14729/) (2023)
- [OmniMotionGPT: Animal Motion Generation with Limited Data](https://mlsanity.com/paper/2311.18303/) (2023)
- [TapMo: Shape-aware Motion Generation of Skeleton-free Characters](https://mlsanity.com/paper/2310.12678/) (2023)
- [Pose-to-Motion: Cross-Domain Motion Retargeting with Pose Prior](https://mlsanity.com/paper/2310.20249/) (2023)
- [A Large-Scale 3D Face Mesh Video Dataset via Neural Re-parameterized Optimization](https://mlsanity.com/paper/2310.03205/) (2023)
- [CharacterMixer: Rig-Aware Interpolation of 3D Characters](https://mlsanity.com/paper/2402.15580/) (2024)

## Built on by (most cited first, 5 of 5)

- [Topology-Agnostic Animal Motion Generation from Text Prompt](https://mlsanity.com/paper/2512.10352/) (2025)
- [MimiCAT: Mimic with Correspondence-Aware Cascade-Transformer for Category-Free 3D Pose Transfer](https://mlsanity.com/paper/2511.18370/) (2025)
- [UniMate: One Unified Model to Animate Diverse Skeletons](https://mlsanity.com/paper/2609.05415/) (2026)
- [Odoriko: A Shape-Aware Multimodal Diffusion Framework for Human Motion](https://mlsanity.com/paper/2606.21135/) (2026)
- [Why Cross-Skeleton Retargeting Is Non-Identifiable: Structural Limits of Generative Motion Models](https://mlsanity.com/paper/2609.37297/) (2026)

## Abstract (arXiv)

Motion synthesis for diverse object categories holds great potential for 3D content creation but remains underexplored due to two key challenges: (1) the lack of comprehensive motion datasets that include a wide range of high-quality motions and annotations, and (2) the absence of methods capable of handling heterogeneous skeletal templates from diverse objects. To address these challenges, we contribute the following: First, we augment the Truebones Zoo dataset, a high-quality animal motion dataset covering over 70 species, by annotating it with detailed text descriptions, making it suitable for text-based motion synthesis. Second, we introduce rig augmentation techniques that generate diverse motion data while preserving consistent dynamics, enabling models to adapt to various skeletal configurations. Finally, we redesign existing motion diffusion models to dynamically adapt to arbitrary skeletal templates, enabling motion synthesis for a diverse range of objects with varying structures. Experiments show that our method learns to generate high-fidelity motions from textual descriptions for diverse and even unseen objects, setting a strong foundation for motion synthesis across diverse object categories and skeletal templates. Qualitative results are available at: hrefhttps://t2m4lvo.github.iohttps://t2m4lvo.github.io.