# A Two-part Transformer Network for Controllable Motion Synthesis

arXiv: https://arxiv.org/abs/2304.12571 · HTML: https://mlsanity.com/paper/2304.12571/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2023.

## Built on by (most cited first, 7 of 7)

- [GUESS:GradUally Enriching SyntheSis for Text-Driven Human Motion Generation](https://mlsanity.com/paper/2401.02142/) (2024)
- [MOST: Motion Diffusion Model for Rare Text via Temporal Clip Banzhaf Interaction](https://mlsanity.com/paper/2507.06590/) (2025)
- [Jointly Understand Your Command and Intention:Reciprocal Co-Evolution between Scene-Aware 3D Human Motion Synthesis and Analysis](https://mlsanity.com/paper/2503.00371/) (2025)
- [Fatigue-PINN: Physics-Informed Fatigue-Driven Motion Modulation and Synthesis](https://mlsanity.com/paper/2502.19056/) (2025)
- [Learning Human Motion from Monocular Videos via Cross-Modal Manifold Alignment](https://mlsanity.com/paper/2404.09499/) (2024)
- [FatigueFusion: Latent Space Fusion for Fatigue-Driven Motion Synthesis](https://mlsanity.com/paper/2604.10199/) (2026)
- [SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergy](https://mlsanity.com/paper/2604.18557/) (2026)

## Abstract (arXiv)

Although part-based motion synthesis networks have been investigated to reduce the complexity of modeling heterogeneous human motions, their computational cost remains prohibitive in interactive applications. To this end, we propose a novel two-part transformer network that aims to achieve high-quality, controllable motion synthesis results in real-time. Our network separates the skeleton into the upper and lower body parts, reducing the expensive cross-part fusion operations, and models the motions of each part separately through two streams of auto-regressive modules formed by multi-head attention layers. However, such a design might not sufficiently capture the correlations between the parts. We thus intentionally let the two parts share the features of the root joint and design a consistency loss to penalize the difference in the estimated root features and motions by these two auto-regressive modules, significantly improving the quality of synthesized motions. After training on our motion dataset, our network can synthesize a wide range of heterogeneous motions, like cartwheels and twists. Experimental and user study results demonstrate that our network is superior to state-of-the-art human motion synthesis networks in the quality of generated motions.