mlsanityData to 14 Oct 2026

Towards Open Domain Text-Driven Synthesis of Multi-Person Motions

arXiv 2405.18483 ↗First posted 28 May 2024cs.CVcited by 29

Not on the map. Why?

Abstract (arXiv)

This work aims to generate natural and diverse group motions of multiple humans from textual descriptions. While single-person text-to-motion generation is extensively studied, it remains challenging to synthesize motions for more than one or two subjects from in-the-wild prompts, mainly due to the lack of available datasets. In this work, we curate human pose and motion datasets by estimating pose information from large-scale image and video datasets. Our models use a transformer-based diffusion framework that accommodates multiple datasets with any number of subjects or frames. Experiments explore both generation of multi-person static poses and generation of multi-person motion sequences. To our knowledge, our method is the first to generate multi-subject motion sequences with high diversity and fidelity from a large variety of textual prompts.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 74 of its 90 references that are in the map. First posted on arXiv 28 May 2024.

Builds on

74 of 90 references.

References · 74

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  5. 2022Video Diffusion Modelscited by 2,305
  6. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  7. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  8. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  9. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  10. 2022Human Motion Diffusion Modelcited by 1,080
  11. 2017End-to-end Recovery of Human Shape and Posecited by 885
  12. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
38 more
  1. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 620
  2. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  3. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
  4. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
  5. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  6. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  7. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
  8. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 408
  9. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 368
  10. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
  11. 2023Human Motion Diffusion as a Generative Priorcited by 357
  12. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 350
  13. 2016The KIT Motion-Language Datasetcited by 338
  14. 2017On human motion prediction using recurrent neural networkscited by 338
  15. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 337
  16. 2018Learning 3D Human Dynamics from Videocited by 316
  17. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 287
  18. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  19. 2021Mesh Graphormercited by 265
  20. 2021BABEL: Bodies, Action and Behavior with English Labelscited by 265
  21. 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
  22. 2022CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimationcited by 238
  23. 2021Robust Motion In-betweeningcited by 229
  24. 2020Monocular, One-stage, Regression of Multiple 3D Peoplecited by 218
  25. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 206
  26. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  27. 2017HP-GAN: Probabilistic 3D human motion prediction via GANcited by 176
  28. 2020History Repeats Itself: Human Motion Prediction via Motion Attentioncited by 165
  29. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 163
  30. 2021Probabilistic Modeling for Human Mesh Recoverycited by 153
  31. 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 153
  32. 2021Putting People in their Place: Monocular Regression of 3D People in Depthcited by 142
  33. 2022Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motioncited by 126
  34. 20214DComplete: Non-Rigid Motion Estimation Beyond the Observable Surfacecited by 124
  35. 2018Human Motion Prediction via Spatio-Temporal Inpaintingcited by 114
  36. 2021SimPoE: Simulated Character Control for 3D Human Pose Estimationcited by 114
  37. 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 91
  38. 2019Human Mesh Recovery from Monocular Images via a Skeleton-disentangled Representationcited by 87

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 12; 2026: 12.

29 papers.

Sort:

By area:

  1. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 84
  2. 2024InterMask: 3D Human Interaction Generation via Collaborative Masked Modelingcited by 47
  3. 2025A Survey on Human Interaction Motion Generationcited by 25
  4. 2025DuetGen: Music Driven Two-Person Dance Generation via Hierarchical Masked Modelingcited by 21
  5. 2024SyncDiff: Synchronized Motion Diffusion for Multi-Body Human-Object Interaction Synthesiscited by 14
  6. 2025SocialGen: Modeling Multi-Human Social Interaction with Language Modelscited by 14
  7. 2025PINO: Person-Interaction Noise Optimization for Long-Duration and Customizable Motion Generation of Arbitrary-Sized Groupscited by 9
  8. 2025Multi-Person Interaction Generation from Two-Person Motion Priorscited by 8
  9. 2025Pulp Motion: Framing-aware multimodal camera and human motion generationcited by 8
  10. 20253D Human Interaction Generation: A Surveycited by 7
  11. 2025Ponimator: Unfolding Interactive Pose for Versatile Human-human Interaction Animationcited by 7
  12. 2024Ig3D: Integrating 3D Face Representations in Facial Expression Inferencecited by 6
17 more
  1. 2025MotionVerse: A Unified Multimodal Framework for Motion Comprehension, Generation and Editingcited by 5
  2. 2025Event-Driven Storytelling with Multiple Lifelike Humans in a 3D Scenecited by 4
  3. 2026Next-Scale Autoregressive Models for Text-to-Motion Generationcited by 4
  4. 2025Leader and Follower: Interactive Motion Generation under Trajectory Constraintscited by 3
  5. 2026Disentangled Hierarchical VAE for 3D Human-Human Interaction Generationcited by 3
  6. 2025Learning to Generate Human-Human-Object Interactions from Textual Descriptionscited by 2
  7. 2024AMG: Avatar Motion Guided Video Generationcited by 1
  8. 2026ReMoGen: Real-time Human Interaction-to-Reaction Generation via Modular Learning from Diverse Datacited by 1
  9. 2026Talking Together: Synthesizing Co-Located 3D Conversations from Audiocited by 0
  10. 2026ConfusionBench: An Expert-Validated Benchmark for Confusion Recognition and Localization in Educational Videoscited by 0
  11. 2026Unified Number-Free Text-to-Motion Generation Via Flow Matchingcited by 0
  12. 2026Paper Reconstruction Evaluation: Evaluating Presentation and Hallucination in AI-written Paperscited by 0
  13. 2026PhysiGen: Integrating Collision-Aware Physical Constraints for High-Fidelity Human-Human Interaction Generationcited by 0
  14. 2026GNOCHI: Generative Neural mOdel for Close Human-Human Interactionscited by 0
  15. 2026A Plug-and-Play 2D Motion Interface for Real-World Motion Language Modelscited by 0
  16. 2026COSMI: COmpositional Synthesis of Multi-object Interactionscited by 0
  17. 2026Scalable Multi-Human Cooperative Manipulation via Commander-Executor Coordinationcited by 0