mlsanityData to 14 Oct 2026

OmniControl: Control Any Joint at Any Time for Human Motion Generation

arXiv 2310.08580 ↗First posted 12 Oct 2023cs.CV cs.GRcited by 238

Not on the map. Why?

Cited by 63 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a novel approach named OmniControl for incorporating flexible spatial control signals into a text-conditioned human motion generation model based on the diffusion process. Unlike previous methods that can only control the pelvis trajectory, OmniControl can incorporate flexible spatial control signals over different joints at different times with only one model. Specifically, we propose analytic spatial guidance that ensures the generated motion can tightly conform to the input control signals. At the same time, realism guidance is introduced to refine all the joints to generate more coherent motion. Both the spatial and realism guidance are essential and they are highly complementary for balancing control accuracy and motion realism. By combining them, OmniControl generates motions that are realistic, coherent, and consistent with the spatial constraints. Experiments on HumanML3D and KIT-ML datasets show that OmniControl not only achieves significant improvement over state-of-the-art methods on pelvis control but also shows promising results when incorporating the constraints over other joints.

Key facts. Cited by 238 papers in the map (data through 14 Oct 2026). Builds on 55 of its 64 references that are in the map. First posted on arXiv 12 Oct 2023.

Builds on

55 of 64 references.

References · 55

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2022Classifier-Free Diffusion Guidancecited by 5,847
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  10. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  11. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  12. 2022Human Motion Diffusion Modelcited by 1,080
38 more
  1. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  2. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  3. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  4. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  5. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
  6. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  7. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 508
  8. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  9. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
  10. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
  11. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  12. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 408
  13. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 368
  14. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
  15. 2023Human Motion Diffusion as a Generative Priorcited by 357
  16. 2016The KIT Motion-Language Datasetcited by 338
  17. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 324
  18. 2022EDGE: Editable Dance Generation From Musiccited by 287
  19. 2019Language2Pose: Natural Language Grounded Pose Forecastingcited by 257
  20. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 239
  21. 2022FLAME: Free-form Language-based Motion Synthesis & Editingcited by 217
  22. 2023Object Motion Guided Human Motion Synthesiscited by 212
  23. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 206
  24. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  25. 2023TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesiscited by 198
  26. 2021Character Controllers Using Motion VAEscited by 184
  27. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  28. 2022HUMANISE: Language-conditioned Human Motion Generation in 3D Scenescited by 158
  29. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
  30. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 138
  31. 2022COUCH: Towards Controllable Human-Chair Interactionscited by 135
  32. 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 124
  33. 2021DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformercited by 111
  34. 2023Synthesizing Diverse Human Motions in 3D Indoor Scenescited by 109
  35. 2022Accelerating Diffusion Models via Early Stop of the Diffusion Processcited by 100
  36. 2022Towards Diverse and Natural Scene-aware 3D Human Motion Synthesiscited by 91
  37. 2022PoseGPT: Quantization-based 3D Human Motion Generation and Forecastingcited by 78
  38. 2023NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesiscited by 78

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 59; 2025: 93; 2026: 76.

238 papers.

Sort:

By area:

  1. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  2. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 168
  3. 2023MMM: Generative Masked Motion Modelcited by 131
  4. 2024MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpaintingcited by 116
  5. 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 98
  6. 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 94
  7. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 93
  8. 2024Multi-Track Timeline Control for Text-Driven 3D Human Motion Generationcited by 91
  9. 2024MotionLLM: Understanding Human Behaviors from Human Motions and Videoscited by 90
  10. 2023EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generationcited by 89
  11. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 85
  12. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 84
38 more
  1. 2024Generating Human Interaction Motions in Scenes with Text Controlcited by 80
  2. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
  3. 2024Flexible Motion In-betweening with Diffusion Modelscited by 71
  4. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 68
  5. 2024CoMo: Controllable Motion Generation through Language Guided Pose Code Editingcited by 67
  6. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  7. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  8. 2024MotionFix: Text-Driven 3D Human Motion Editingcited by 64
  9. 2023ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactionscited by 62
  10. 2024MotionGPT-2: A General-Purpose Motion-Language Model for Motion Generation and Understandingcited by 59
  11. 2024Taming Diffusion Probabilistic Models for Character Controlcited by 57
  12. 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 53
  13. 2025TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenizationcited by 51
  14. 2024StableMoFusion: Towards Robust and Efficient Diffusion-based Motion Generation Frameworkcited by 50
  15. 2024Generating Human Motion in 3D Scenes from Text Descriptionscited by 49
  16. 2025Inference-time Scaling of Diffusion Models through Classical Searchcited by 49
  17. 2023OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllerscited by 47
  18. 2024Motion-Agent: A Conversational Framework for Human Motion Generation with LLMscited by 47
  19. 2025FullDiT: Multi-Task Video Generative Foundation Model with Full Attentioncited by 47
  20. 2024THOR: Text to Human-Object Interaction Diffusion via Relation Interventioncited by 46
  21. 2024The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motioncited by 46
  22. 2025ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Modelscited by 43
  23. 2024FreeMotion: A Unified Framework for Number-free Text-to-Motion Synthesiscited by 42
  24. 2025AnyTop: Character Animation Diffusion with Any Topologycited by 41
  25. 2025SALAD: Skeleton-aware Latent Diffusion for Text-driven Motion Generation and Editingcited by 39
  26. 2024MaskControl: Spatio-Temporal Control for Masked Motion Synthesiscited by 38
  27. 2024SMooDi: Stylized Motion Diffusion Modelcited by 37
  28. 2026Kimodo: Scaling Controllable Human Motion Generationcited by 37
  29. 2024Enabling Synergistic Full-Body Control in Prompt-Based Co-Speech Motion Generationcited by 35
  30. 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 32
  31. 2024Monkey See, Monkey Do: Harnessing Self-attention in Motion Diffusion for Zero-shot Motion Transfercited by 31
  32. 2024Unimotion: Unifying 3D Human Motion Synthesis and Understandingcited by 30
  33. 2025MotionLab: Unified Human Motion Generation and Editing via the Motion-Condition-Motion Paradigmcited by 30
  34. 2024Learning from Massive Human Videos for Universal Humanoid Pose Controlcited by 29
  35. 2025Absolute Coordinates Make Motion Generation Easycited by 29
  36. 2025Ready-to-React: Online Reaction Policy for Two-Character Interaction Generationcited by 28
  37. 2025SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Predictioncited by 27
  38. 2024ReinDiffuse: Crafting Physically Plausible Motions with Reinforced Diffusion Modelcited by 26

50 most cited of 238 All 238 (JSON)