UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priors
Not on the map. Why?
Abstract (arXiv)
Large-scale foundation models (LFMs) have recently made impressive progress in text-to-motion generation by learning strong generative priors from massive 3D human motion datasets and paired text descriptions. However, how to effectively and efficiently leverage such single-purpose motion LFMs, i.e., text-to-motion synthesis, in more diverse cross-modal and in-context motion generation downstream tasks remains largely unclear. Prior work typically adapts pretrained generative priors to individual downstream tasks in a task-specific manner. In contrast, our goal is to unlock such priors to support a broad spectrum of downstream motion generation tasks within a single unified framework. To bridge this gap, we present UMO, a simple yet general unified formulation that casts diverse downstream tasks into compositions of atomic per-frame operations, enabling in-context adaptation to unlock the generative priors of pretrained DiT-based motion LFMs. Specifically, UMO introduces three learnable frame-level meta-operation embeddings to specify per-frame intent and employs lightweight temporal fusion to inject in-context cues into the pretrained backbone, with negligible runtime overhead compared to the base model. With this design, UMO finetunes the pretrained model, originally limited to text-to-motion generation, to support diverse previously unsupported tasks, including temporal inpainting, text-guided motion editing, text-serialized geometric constraints, and multi-identity reaction generation. Experiments demonstrate that UMO consistently outperforms task-specific and training-free baselines across a wide range of benchmarks, despite using a single unified model. Code and model will be publicly available. Project Page: https://oliver-cong02.github.io/UMO.github.io/
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 77 of its 78 references that are in the map. First posted on arXiv 16 Mar 2026.
Builds on
77 of 78 references.
References · 77
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
- 2025Qwen3 Technical Reportcited by 8,195
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
- 2022Human Motion Diffusion Modelcited by 1,062
38 more
- 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
- 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
- 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
- 2025Qwen3-Omni Technical Reportcited by 451
- 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
- 2023Human Motion Diffusion as a Generative Priorcited by 354
- 2023An Edit Friendly DDPM Noise Space: Inversion and Manipulationscited by 243
- 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
- 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 233
- 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
- 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 205
- 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 181
- 2024Scaling Up Dynamic Human-Scene Interaction Modelingcited by 137
- 2023AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanismcited by 116
- 2024CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character controlcited by 103
- 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
- 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 97
- 2025ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Fillingcited by 95
- 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 93
- 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 89
- 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generationcited by 88
- 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
- 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 82
- 2024Flexible Motion In-betweening with Diffusion Modelscited by 69
- 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 67
- 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 67
- 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 63
- 2024MotionFix: Text-Driven 3D Human Motion Editingcited by 63
- 2024Taming Diffusion Probabilistic Models for Character Controlcited by 56
- 2024ACE: All-round Creator and Editor Following Instructions via Diffusion Transformercited by 56
- 2024ReGenNet: Towards Human Action-Reaction Synthesiscited by 53
- 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 52
- 2024InterMask: 3D Human Interaction Generation via Collaborative Masked Modelingcited by 47
- 2024The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motioncited by 46
- 2025SnapMoGen: Human Motion Generation from Expressive Textscited by 44
- 2025GENMO: A GENeralist Model for Human MOtioncited by 43
- 2024FreeMotion: A Unified Framework for Number-free Text-to-Motion Synthesiscited by 42
- 2024M3GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generationcited by 37
50 most cited of 77 All 77 (JSON)
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026HumanCLAW: Can Vision-Language Models Act Through a Body?cited by 1
- 2026UniMate: One Unified Model to Animate Diverse Skeletonscited by 0
- 2026MotionSpaceFlow: Representation-Aware Flow Matching in Direct Motion Spacecited by 0
- 2026MotionMaestro: Masked Tokenization for Unified Motion Generationcited by 0
- 2026World Motion Models: Flexible Sequence Modeling of SE(3) Trajectoriescited by 0
- 2026World Motion Models: Flexible Sequence Modeling of SE(3) Trajectories1 Oct 2026
- 2026MotionSpaceFlow: Representation-Aware Flow Matching in Direct Motion Space28 Sep 2026
- 2026MotionMaestro: Masked Tokenization for Unified Motion Generation26 Sep 2026
- 2026UniMate: One Unified Model to Animate Diverse Skeletons4 Sep 2026
- 2026HumanCLAW: Can Vision-Language Models Act Through a Body?29 Jul 2026