# SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generation

arXiv: https://arxiv.org/abs/2304.10417 · HTML: https://mlsanity.com/paper/2304.10417/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Builds on 56 of its 76 references that are in the map.
- First posted on arXiv 20 Apr 2023.

## Builds on (most cited first, 30 of 56)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [Action-Conditioned 3D Human Motion Synthesis with Transformer VAE](https://mlsanity.com/paper/2104.05670/) (2021)
- [TEMOS: Generating diverse human motions from textual descriptions](https://mlsanity.com/paper/2204.14109/) (2022)
- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [Action2Motion: Conditioned Generation of 3D Human Motions](https://mlsanity.com/paper/2007.15240/) (2020)
- [MotionCLIP: Exposing Human Motion Generation to CLIP Space](https://mlsanity.com/paper/2203.08063/) (2022)
- [Learning joint reconstruction of hands and manipulated objects](https://mlsanity.com/paper/1904.05767/) (2019)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts](https://mlsanity.com/paper/2207.01696/) (2022)
- [The KIT Motion-Language Dataset](https://mlsanity.com/paper/1607.03827/) (2016)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [BABEL: Bodies, Action and Behavior with English Labels](https://mlsanity.com/paper/2106.09696/) (2021)
- [Language2Pose: Natural Language Grounded Pose Forecasting](https://mlsanity.com/paper/1907.01108/) (2019)
- [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021)
- [MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis](https://mlsanity.com/paper/2212.04495/) (2022)
- [FLAME: Free-form Language-based Motion Synthesis & Editing](https://mlsanity.com/paper/2209.00349/) (2022)
- [AGORA: Avatars in Geography Optimized for Regression Analysis](https://mlsanity.com/paper/2104.14643/) (2021)
- [Stochastic Scene-Aware Motion Prediction](https://mlsanity.com/paper/2108.08284/) (2021)

## Built on by (most cited first, 30 of 64)

- [TMR: Text-to-Motion Retrieval Using Contrastive 3D Human Motion Synthesis](https://mlsanity.com/paper/2305.00976/) (2023)
- [Unified Human-Scene Interaction via Prompted Chain-of-Contacts](https://mlsanity.com/paper/2309.07918/) (2023)
- [Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation](https://mlsanity.com/paper/2401.08559/) (2024)
- [Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordance](https://mlsanity.com/paper/2403.18036/) (2024)
- [CoMo: Controllable Motion Generation through Language Guided Pose Code Editing](https://mlsanity.com/paper/2403.13900/) (2024)
- [Large Motion Model for Unified Multi-Modal Motion Generation](https://mlsanity.com/paper/2404.01284/) (2024)
- [InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction](https://mlsanity.com/paper/2403.19652/) (2024)
- [MotionFix: Text-Driven 3D Human Motion Editing](https://mlsanity.com/paper/2408.00712/) (2024)
- [FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models](https://mlsanity.com/paper/2312.08459/) (2023)
- [ParCo: Part-Coordinating Text-to-Motion Synthesis](https://mlsanity.com/paper/2403.18512/) (2024)
- [Generating Human Motion in 3D Scenes from Text Descriptions](https://mlsanity.com/paper/2405.07784/) (2024)
- [THOR: Text to Human-Object Interaction Diffusion via Relation Intervention](https://mlsanity.com/paper/2403.11208/) (2024)
- [Iterative Motion Editing with Natural Language](https://mlsanity.com/paper/2312.11538/) (2023)
- [Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion](https://mlsanity.com/paper/2312.04466/) (2023)
- [Enabling Synergistic Full-Body Control in Prompt-Based Co-Speech Motion Generation](https://mlsanity.com/paper/2410.00464/) (2024)
- [Monkey See, Monkey Do: Harnessing Self-attention in Motion Diffusion for Zero-shot Motion Transfer](https://mlsanity.com/paper/2406.06508/) (2024)
- [MotionLab: Unified Human Motion Generation and Editing via the Motion-Condition-Motion Paradigm](https://mlsanity.com/paper/2502.02358/) (2025)
- [in2IN: Leveraging individual Information to Generate Human INteractions](https://mlsanity.com/paper/2404.09988/) (2024)
- [IMUGPT 2.0: Language-Based Cross Modality Transfer for Sensor-Based Human Activity Recognition](https://mlsanity.com/paper/2402.01049/) (2024)
- [MotionScript: Natural Language Descriptions for Expressive 3D Human Motions](https://mlsanity.com/paper/2312.12634/) (2023)
- [Fg-T2M++: LLMs-Augmented Fine-Grained Text Driven Human Motion Generation](https://mlsanity.com/paper/2502.05534/) (2025)
- [HUMOS: Human Motion Model Conditioned on Body Shape](https://mlsanity.com/paper/2409.03944/) (2024)
- [Language-Driven Interactive Traffic Trajectory Generation](https://mlsanity.com/paper/2405.15388/) (2024)
- [Pay Attention and Move Better: Harnessing Attention for Interactive Motion Generation and Training-free Editing](https://mlsanity.com/paper/2410.18977/) (2024)
- [A Cross-Dataset Study for Text-based 3D Human Motion Retrieval](https://mlsanity.com/paper/2405.16909/) (2024)
- [UniEgoMotion: A Unified Model for Egocentric Motion Reconstruction, Forecasting, and Generation](https://mlsanity.com/paper/2508.01126/) (2025)
- [PuzzleAvatar: Assembling 3D Avatars from Personal Albums](https://mlsanity.com/paper/2405.14869/) (2024)
- [Dynamic Motion Blending for Versatile Motion Editing](https://mlsanity.com/paper/2503.20724/) (2025)
- [Bridging the Gap between Human Motion and Action Semantics via Kinematic Phrases](https://mlsanity.com/paper/2310.04189/) (2023)
- [MixerMDM: Learnable Composition of Human Motion Diffusion Models](https://mlsanity.com/paper/2504.01019/) (2025)

## Abstract (arXiv)

Our goal is to synthesize 3D human motions given textual inputs describing simultaneous actions, for example 'waving hand' while 'walking' at the same time. We refer to generating such simultaneous movements as performing 'spatial compositions'. In contrast to temporal compositions that seek to transition from one action to another, spatial compositing requires understanding which body parts are involved in which action, to be able to move them simultaneously. Motivated by the observation that the correspondence between actions and body parts is encoded in powerful language models, we extract this knowledge by prompting GPT-3 with text such as "what are the body parts involved in the action <action name>?", while also providing the parts list and few-shot examples. Given this action-part mapping, we combine body parts from two motions together and establish the first automated method to spatially compose two actions. However, training data with compositional actions is always limited by the combinatorics. Hence, we further create synthetic data with this approach, and use it to train a new state-of-the-art text-to-motion generation model, called SINC ("SImultaneous actioN Compositions for 3D human motions"). In our experiments, that training with such GPT-guided synthetic data improves spatial composition generation over baselines. Our code is publicly available at https://sinc.is.tue.mpg.de/.