mlsanityData to 7 Oct 2026

UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priors

arXiv 2603.15975 ↗First posted 16 Mar 2026cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Large-scale foundation models (LFMs) have recently made impressive progress in text-to-motion generation by learning strong generative priors from massive 3D human motion datasets and paired text descriptions. However, how to effectively and efficiently leverage such single-purpose motion LFMs, i.e., text-to-motion synthesis, in more diverse cross-modal and in-context motion generation downstream tasks remains largely unclear. Prior work typically adapts pretrained generative priors to individual downstream tasks in a task-specific manner. In contrast, our goal is to unlock such priors to support a broad spectrum of downstream motion generation tasks within a single unified framework. To bridge this gap, we present UMO, a simple yet general unified formulation that casts diverse downstream tasks into compositions of atomic per-frame operations, enabling in-context adaptation to unlock the generative priors of pretrained DiT-based motion LFMs. Specifically, UMO introduces three learnable frame-level meta-operation embeddings to specify per-frame intent and employs lightweight temporal fusion to inject in-context cues into the pretrained backbone, with negligible runtime overhead compared to the base model. With this design, UMO finetunes the pretrained model, originally limited to text-to-motion generation, to support diverse previously unsupported tasks, including temporal inpainting, text-guided motion editing, text-serialized geometric constraints, and multi-identity reaction generation. Experiments demonstrate that UMO consistently outperforms task-specific and training-free baselines across a wide range of benchmarks, despite using a single unified model. Code and model will be publicly available. Project Page: https://oliver-cong02.github.io/UMO.github.io/

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 77 of its 78 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

77 of 78 references.

References · 77

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2022Scalable Diffusion Models with Transformerscited by 5,931
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2022Flow Matching for Generative Modelingcited by 5,426
  7. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  8. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  9. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  10. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  11. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  12. 2022Human Motion Diffusion Modelcited by 1,062
38 more
  1. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
  2. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
  3. 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
  4. 2025Qwen3-Omni Technical Reportcited by 451
  5. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
  6. 2023Human Motion Diffusion as a Generative Priorcited by 354
  7. 2023An Edit Friendly DDPM Noise Space: Inversion and Manipulationscited by 243
  8. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 241
  9. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 233
  10. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  11. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 205
  12. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 181
  13. 2024Scaling Up Dynamic Human-Scene Interaction Modelingcited by 137
  14. 2023AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanismcited by 116
  15. 2024CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character controlcited by 103
  16. 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
  17. 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 97
  18. 2025ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Fillingcited by 95
  19. 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 93
  20. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 89
  21. 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generationcited by 88
  22. 2025UniVideo: Unified Understanding, Generation, and Editing for Videoscited by 85
  23. 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 82
  24. 2024Flexible Motion In-betweening with Diffusion Modelscited by 69
  25. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 67
  26. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 67
  27. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 63
  28. 2024MotionFix: Text-Driven 3D Human Motion Editingcited by 63
  29. 2024Taming Diffusion Probabilistic Models for Character Controlcited by 56
  30. 2024ACE: All-round Creator and Editor Following Instructions via Diffusion Transformercited by 56
  31. 2024ReGenNet: Towards Human Action-Reaction Synthesiscited by 53
  32. 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 52
  33. 2024InterMask: 3D Human Interaction Generation via Collaborative Masked Modelingcited by 47
  34. 2024The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motioncited by 46
  35. 2025SnapMoGen: Human Motion Generation from Expressive Textscited by 44
  36. 2025GENMO: A GENeralist Model for Human MOtioncited by 43
  37. 2024FreeMotion: A Unified Framework for Number-free Text-to-Motion Synthesiscited by 42
  38. 2024M3GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generationcited by 37

50 most cited of 77 All 77 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.