# Listen to Dance: Music-driven choreography generation using Autoregressive Encoder-Decoder Network

arXiv: https://arxiv.org/abs/1811.00818 · HTML: https://mlsanity.com/paper/1811.00818/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2018.

## Built on by (most cited first, 11 of 11)

- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformer](https://mlsanity.com/paper/2103.10206/) (2021)
- [TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration](https://mlsanity.com/paper/2304.02419/) (2023)
- [Single Motion Diffusion](https://mlsanity.com/paper/2302.05905/) (2023)
- [MoDi: Unconditional Motion Synthesis from Diverse Data](https://mlsanity.com/paper/2206.08010/) (2022)
- [DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation](https://mlsanity.com/paper/2308.02915/) (2023)
- [MusicFace: Music-driven Expressive Singing Face Synthesis](https://mlsanity.com/paper/2303.14044/) (2023)
- [Robust Dancer: Long-term 3D Dance Synthesis Using Unpaired Data](https://mlsanity.com/paper/2303.16856/) (2023)
- [TokenDance: Token-to-Token Music-to-Dance Generation with Bidirectional Mamba](https://mlsanity.com/paper/2603.27314/) (2026)
- [May the Dance be with You: Dance Generation Framework for Non-Humanoids](https://mlsanity.com/paper/2405.19743/) (2024)
- [How Animals Dance (When You're Not Looking)](https://mlsanity.com/paper/2505.23738/) (2025)

## Abstract (arXiv)

Automatic choreography generation is a challenging task because it often requires an understanding of two abstract concepts - music and dance - which are realized in the two different modalities, namely audio and video, respectively. In this paper, we propose a music-driven choreography generation system using an auto-regressive encoder-decoder network. To this end, we first collect a set of multimedia clips that include both music and corresponding dance motion. We then extract the joint coordinates of the dancer from video and the mel-spectrogram of music from audio, and train our network using music-choreography pairs as input. Finally, a novel dance motion is generated at the inference time when only music is given as an input. We performed a user study for a qualitative evaluation of the proposed method, and the results show that the proposed model is able to generate musically meaningful and natural dance movements given an unheard song.