mlsanityData to 14 Oct 2026

Flexible Motion In-betweening with Diffusion Models

arXiv 2405.11126 ↗First posted 17 May 2024cs.CV cs.GR cs.LGcited by 71

On the map: Diffusion models → Video generation

Abstract (arXiv)

Motion in-betweening, a fundamental task in character animation, consists of generating motion sequences that plausibly interpolate user-provided keyframe constraints. It has long been recognized as a labor-intensive and challenging process. We investigate the potential of diffusion models in generating diverse human motions guided by keyframes. Unlike previous inbetweening methods, we propose a simple unified model capable of generating precise and diverse motions that conform to a flexible range of user-specified spatial constraints, as well as text conditioning. To this end, we propose Conditional Motion Diffusion In-betweening (CondMDI) which allows for arbitrary dense-or-sparse keyframe placement and partial keyframe constraints while generating high-quality motions that are diverse and coherent with the given keyframes. We evaluate the performance of CondMDI on the text-conditioned HumanML3D dataset and demonstrate the versatility and efficacy of diffusion models for keyframe in-betweening. We further explore the use of guidance and imputation-based approaches for inference-time keyframing and compare CondMDI against these methods.

Key facts. Cited by 71 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 40 of its 74 references that are in the map. First posted on arXiv 17 May 2024.

Builds on

40 of 74 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023Guided Motion Diffusion for Controllable Human Motion Synthesis21 May
  2. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions12 Apr
  3. 2023Human Motion Diffusion as a Generative Prior2 Mar
  4. 2023Single Motion Diffusion12 Feb
  5. 2022Executing your Commands via Motion Diffusion in Latent Space8 Dec
  6. 2022PhysDiff: Physics-Guided Human Motion Diffusion Model5 Dec
  7. 2022Human Motion Diffusion Model29 Sep
  8. 2022FLAME: Free-form Language-based Motion Synthesis & Editing1 Sep
  9. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model31 Aug
  10. 2022Flexible Diffusion Modeling of Long Videos23 May

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2014Going Deeper with Convolutionscited by 4,241
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  11. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  12. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
18 more
  1. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  2. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  3. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 238
  4. 2021Robust Motion In-betweeningcited by 229
  5. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  6. 2021Character Controllers Using Motion VAEscited by 184
  7. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 178
  8. 2017Text2Action: Generative Adversarial Synthesis from Language to Actioncited by 106
  9. 2021Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agentscited by 106
  10. 2019Structured Prediction Helps 3D Human Motion Modellingcited by 96
  11. 2022NeMF: Neural Motion Fields for Kinematic Animationcited by 68
  12. 2017Learning Human Motion Models for Long-term Predictionscited by 67
  13. 2021Single-Shot Motion Completion with Transformercited by 59
  14. 2018Recurrent Transition Networks for Character Locomotioncited by 52
  15. 2019MoGlow: Probabilistic and controllable motion synthesis using normalising flowscited by 47
  16. 2023MAS: Multi-view Ancestral Sampling for 3D motion generation using 2D diffusioncited by 37
  17. 2021Task-Generic Hierarchical Human Motion Prior using VAEscited by 33
  18. 2020Generative Tweening: Long-term Inbetweening of 3D Human Motionscited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 35; 2026: 26.

71 papers.

Sort:

By area:

  1. 2025BeyondMimic: From Motion Tracking to Versatile Humanoid Control via Guided Diffusioncited by 165
  2. 2024CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character controlcited by 109
  3. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 84
  4. 2024MotionFix: Text-Driven 3D Human Motion Editingcited by 64
  5. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 59
  6. 2025GENMO: A GENeralist Model for Human MOtioncited by 49
  7. 2024MaskControl: Spatio-Temporal Control for Masked Motion Synthesiscited by 38
  8. 2024SMooDi: Stylized Motion Diffusion Modelcited by 37
  9. 2025MotionGPT3: Human Motion as a Second Modalitycited by 37
  10. 2026Kimodo: Scaling Controllable Human Motion Generationcited by 37
  11. 2025HOIGPT: Learning Long Sequence Hand-Object Interaction with Language Modelscited by 33
  12. 2024Monkey See, Monkey Do: Harnessing Self-attention in Motion Diffusion for Zero-shot Motion Transfercited by 31
38 more
  1. 2025MotionLab: Unified Human Motion Generation and Editing via the Motion-Condition-Motion Paradigmcited by 30
  2. 2025Diffuse-CLoC: Guided Diffusion for Physics-based Character Look-ahead Controlcited by 28
  3. 2025PARC: Physics-based Augmentation with Reinforcement Learning for Character Controllerscited by 28
  4. 2024Pay Attention and Move Better: Harnessing Attention for Interactive Motion Generation and Training-free Editingcited by 18
  5. 2025HOIDiNi: Human-Object Interaction through Diffusion Noise Optimizationcited by 18
  6. 2025Sketch2Anim: Towards Transferring Sketch Storyboards into 3D Animationcited by 17
  7. 2025Dance Like a Chicken: Low-Rank Stylization for Human Motion Diffusioncited by 16
  8. 2025SceneMI: Motion In-betweening for Modeling Human-Scene Interactionscited by 11
  9. 2025Motion Synthesis with Sparse and Flexible Keyjoint Controlcited by 9
  10. 2025Multi-Person Interaction Generation from Two-Person Motion Priorscited by 8
  11. 2025Motion Generation: A Survey of Generative Approaches and Benchmarkscited by 8
  12. 2025In-2-4D: Inbetweening from Two Single-View Images to 4D Generationcited by 7
  13. 2026ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generationcited by 7
  14. 2025StableMotion: Training Motion Cleanup Models with Unpaired Corrupted Datacited by 6
  15. 2025Towards Robust and Controllable Text-to-Motion via Masked Autoregressive Diffusioncited by 6
  16. 2026MotionBricks: Scalable Real-Time Motions with Modular Latent Generative Model and Smart Primitivescited by 6
  17. 2024A Plug-and-Play Physical Motion Restoration Approach for In-the-Wild High-Difficulty Motionscited by 5
  18. 2025Waymo-3DSkelMo: A Multi-Agent 3D Skeletal Motion Dataset for Pedestrian Interaction Modeling in Autonomous Drivingcited by 5
  19. 2026ProjFlow: Projection Sampling with Flow Matching for Zero-Shot Exact Spatial Motion Controlcited by 5
  20. 2026UMO: Unified In-Context Learning Unlocks Motion Foundation Model Priorscited by 5
  21. 2024Real-time Diverse Motion In-betweening with Space-time Controlcited by 4
  22. 2025Generative Motion Infilling From Imprecisely Timed Keyframescited by 4
  23. 2025AnyMoLe: Any Character Motion In-betweening Leveraging Video Diffusion Modelscited by 4
  24. 2025FlowMotion: Target-Predictive Conditional Flow Matching for Jitter-Reduced Text-Driven Human Motion Generationcited by 4
  25. 2025Dyadic Mamba: Long-term Dyadic Human Motion Synthesiscited by 4
  26. 2025OmniMotion-X: Versatile Multimodal Whole-Body Motion Generationcited by 4
  27. 2025HMVLM: Human Motion-Vision-Lanuage Model via MoE LoRAcited by 4
  28. 2025SPG: Improving Motion Diffusion by Smooth Perturbation Guidancecited by 2
  29. 2025ExGes: Expressive Human Motion Retrieval and Modulation for Audio-Driven Gesture Synthesiscited by 2
  30. 2025Automatic Generation of Aerobatic Flight in Complex Environments via Diffusion Modelscited by 2
  31. 2025E-React: Towards Emotionally Controlled Synthesis of Human Reactionscited by 2
  32. 2026SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learningcited by 2
  33. 2026RoMo: A Large-Scale, Richly Organized Dataset and Semantic Taxonomy for Human Motion Generationcited by 2
  34. 2025A Plug-and-Play Multi-Criteria Guidance for Diverse In-Betweening Human Motion Generationcited by 1
  35. 2025Controllable Long-term Motion Generation with Extended Joint Targetscited by 1
  36. 2026Sketch2Colab: Sketch-Conditioned Multi-Human Animation via Controllable Flow Distillationcited by 1
  37. 2026InfiniteDance: Scalable 3D Dance Generation Towards in-the-wild Generalizationcited by 1
  38. 2026KV-Control: Parameter-Efficient K/V Injection for Trajectory-Controlled Text-to-Motioncited by 1

50 most cited of 71 All 71 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Less is More: Improving Motion Diffusion Models with Sparse Keyframes
  2. 2023Enhanced Fine-grained Motion Diffusion for Text-driven Human Motion Synthesis
  3. 2025Scalable Motion In-betweening via Diffusion and Physics-Based Character Adaptation
  4. 2024Human Motion Synthesis_ A Diffusion Approach for Motion Stitching and In-Betweening
  5. 2023DiffusionPhase: Motion Diffusion in Frequency Domain
  6. 2024Towards motion from video diffusion models
  7. 2024Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolation
  8. 2026Diffusion Path Alignment for Long-Range Motion Generation and Domain Transitions