mlsanityData to 14 Oct 2026

IKMo: Image-Keyframed Motion Generation with Trajectory-Pose Conditioned Motion Diffusion Model

arXiv 2505.21146 ↗First posted 27 May 2025cs.GR cs.CVcited by 0

On the map: Diffusion models → Video generation

Abstract (arXiv)

While text-to-motion generation is mainstream, integrating explicit spatial constraints has become a standard solution for precise animation. Existing state-of-the-art models are tailored for professional workflows, effectively processing high-quality, pre-aligned 3D inputs via unified conditioning mechanisms. However, this strictly aligned setting restricts accessibility for general users who rely on unaligned, in-the-wild 2D reference images. This accessible scenario introduces a fundamental conflict: heterogeneous geometric constraints, where user-defined global trajectories clash with local image-derived poses. To resolve this coordinate mismatch, we propose TPMDM, a diffusion model featuring a decoupled parallel control strategy that explicitly isolates and aligns these divergent signals. Furthermore, we introduce IKMo, a holistic framework that synergizes TPMDM with a VLM-powered multi-agent system. By bridging the semantic-geometric gap, IKMo translates user-friendly inputs into explicit motion constraints to drive the diffusion backbone. Experiments on HumanML3D and KIT-ML demonstrate that our method establishes a new state-of-the-art in the task of generating motions under heterogeneous trajectory and keyframe pose constraints.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 46 of its 64 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

46 of 64 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023TrailBlazer: Trajectory Control for Diffusion-Based Video Generation31 Dec
  2. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation28 Nov
  3. 2023Guided Motion Diffusion for Controllable Human Motion Synthesis21 May
  4. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions12 Apr
  5. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model3 Apr
  6. 2023Human Motion Diffusion as a Generative Prior2 Mar
  7. 2022FLAME: Free-form Language-based Motion Synthesis & Editing1 Sep
  8. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model31 Aug

Other references · 38

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  4. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  5. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  6. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
  7. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  8. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 408
  9. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
  10. 2016The KIT Motion-Language Datasetcited by 338
  11. 2018Everybody Dance Nowcited by 315
  12. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 314
26 more
  1. 2022EDGE: Editable Dance Generation From Musiccited by 287
  2. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 239
  3. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 238
  4. 2023TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesiscited by 198
  5. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  6. 2022Generating Holistic 3D Human Motion from Speechcited by 180
  7. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 138
  8. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 138
  9. 2022COUCH: Towards Controllable Human-Chair Interactionscited by 135
  10. 2021DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformercited by 111
  11. 2023Synthesizing Diverse Human Motions in 3D Indoor Scenescited by 109
  12. 2024Boximator: Generating Rich and Controllable Motions for Video Synthesiscited by 107
  13. 2023EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modelingcited by 105
  14. 2024TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representationcited by 95
  15. 2023Unified Human-Scene Interaction via Prompted Chain-of-Contactscited by 94
  16. 2024MotionLLM: Understanding Human Behaviors from Human Motions and Videoscited by 90
  17. 2020Convolutional Autoencoders for Human Motion Infillingcited by 73
  18. 2024Autonomous Character-Scene Interaction Synthesis from Text Instructioncited by 56
  19. 2024CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamicscited by 44
  20. 2023Digital Life Project: Autonomous 3D Characters with Social Intelligencecited by 43
  21. 2023Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusioncited by 39
  22. 2024Enabling Synergistic Full-Body Control in Prompt-Based Co-Speech Motion Generationcited by 35
  23. 2023Revisit Human-Scene Interaction via Space Occupancycited by 25
  24. 2022Action-conditioned On-demand Motion Generationcited by 18
  25. 2024InterFusion: Text-Driven Generation of 3D Human-Object Interactioncited by 18
  26. 2025StickMotion: Generating 3D Human Motions by Drawing a Stickmancited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Strong and Controllable 3D Motion Generation
  2. 2024Move-in-2D: 2D-Conditioned Human Motion Generation
  3. 2024StableMoFusion: Towards Robust and Efficient Diffusion-based Motion Generation Framework
  4. 2025RealisMotion: Decomposed Human Motion Control and Video Generation in the World Space
  5. 2026Coordinating Multiple Conditions for Trajectory-Controlled Human Motion Generation
  6. 2026MotionMaestro: Masked Tokenization for Unified Motion Generation
  7. 2024KMM: Key Frame Mask Mamba for Extended Motion Generation
  8. 2022Human Motion Diffusion Model