# Dancing Points: Synthesizing Ballroom Dancing with Three-Point Inputs

arXiv: https://arxiv.org/abs/2601.02096 · HTML: https://mlsanity.com/paper/2601.02096/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 42 references that are in the map.
- First posted on arXiv 5 Jan 2026.

## Builds on (most cited first, 27 of 27)

- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021)
- [InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions](https://mlsanity.com/paper/2304.05684/) (2023)
- [Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models](https://mlsanity.com/paper/2211.09707/) (2022)
- [Character Controllers Using Motion VAEs](https://mlsanity.com/paper/2103.14274/) (2021)
- [AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensing](https://mlsanity.com/paper/2207.13784/) (2022)
- [Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model](https://mlsanity.com/paper/2304.08577/) (2023)
- [ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactions](https://mlsanity.com/paper/2311.17057/) (2023)
- [Taming Diffusion Probabilistic Models for Character Control](https://mlsanity.com/paper/2404.15121/) (2024)
- [ReGenNet: Towards Human Action-Reaction Synthesis](https://mlsanity.com/paper/2403.11882/) (2024)
- [RoHM: Robust Human Motion Reconstruction via Diffusion](https://mlsanity.com/paper/2401.08570/) (2024)
- [MoGlow: Probabilistic and controllable motion synthesis using normalising flows](https://mlsanity.com/paper/1905.06598/) (2019)
- [Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompaniment](https://mlsanity.com/paper/2403.18811/) (2024)
- [InterMask: 3D Human Interaction Generation via Collaborative Masked Modeling](https://mlsanity.com/paper/2410.10010/) (2024)
- [LoBSTr: Real-time Lower-body Pose Prediction from Sparse Upper-body Tracking Signals](https://mlsanity.com/paper/2103.01500/) (2021)
- [BoDiffusion: Diffusing Sparse Observations for Full-Body Human Motion Synthesis](https://mlsanity.com/paper/2304.11118/) (2023)
- [Interaction Transformer for Human Reaction Generation](https://mlsanity.com/paper/2207.01685/) (2022)
- [Simulation and Retargeting of Complex Multi-Character Interactions](https://mlsanity.com/paper/2305.20041/) (2023)
- [QuestEnvSim: Environment-Aware Simulated Motion Tracking from Sparse Sensors](https://mlsanity.com/paper/2306.05666/) (2023)
- [Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation](https://mlsanity.com/paper/2502.20370/) (2025)
- [Motion In-Betweening with Phase Manifolds](https://mlsanity.com/paper/2308.12751/) (2023)
- [SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data](https://mlsanity.com/paper/2311.02191/) (2023)
- [DuetGen: Music Driven Two-Person Dance Generation via Hierarchical Masked Modeling](https://mlsanity.com/paper/2506.18680/) (2025)
- [Towards Immersive Human-X Interaction: A Real-Time Framework for Physically Plausible Motion Synthesis](https://mlsanity.com/paper/2508.02106/) (2025)
- [Combining Motion Matching and Orientation Prediction to Animate Avatars for Consumer-Grade VR Devices](https://mlsanity.com/paper/2209.11478/) (2022)
- [Think-Then-React: Towards Unconstrained Human Action-to-Reaction Generation](https://mlsanity.com/paper/2503.16451/) (2025)

## Abstract (arXiv)

Ballroom dancing is a structured yet expressive motion category. Its highly diverse movement and complex interactions between leader and follower dancers make the understanding and synthesis challenging. We demonstrate that the three-point trajectory available from a virtual reality (VR) device can effectively serve as a dancer's motion descriptor, simplifying the modeling and synthesis of interplay between dancers' full-body motions down to sparse trajectories. Thanks to the low dimensionality, we can employ an efficient MLP network to predict the follower's three-point trajectory directly from the leader's three-point input for certain types of ballroom dancing, addressing the challenge of modeling high-dimensional full-body interaction. It also prevents our method from overfitting thanks to its compact yet explicit representation. By leveraging the inherent structure of the movements and carefully planning the autoregressive procedure, we show a deterministic neural network is able to translate three-point trajectories into a virtual embodied avatar, which is typically considered under-constrained and requires generative models for common motions. In addition, we demonstrate this deterministic approach generalizes beyond small, structured datasets like ballroom dancing, and performs robustly on larger, more diverse datasets such as LaFAN. Our method provides a computationally- and data-efficient solution, opening new possibilities for immersive paired dancing applications. Code and pre-trained models for this paper are available at https://peizhuoli.github.io/dancing-points.