# Weakly Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation

arXiv: https://arxiv.org/abs/1807.01126 · HTML: https://mlsanity.com/paper/1807.01126/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2018.

## Built on by (most cited first, 22 of 22)

- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory](https://mlsanity.com/paper/2203.13055/) (2022)
- [TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration](https://mlsanity.com/paper/2304.02419/) (2023)
- [Music-Driven Group Choreography](https://mlsanity.com/paper/2303.12337/) (2023)
- [AI-Generated Content (AIGC) for Various Data Modalities: A Survey](https://mlsanity.com/paper/2308.14177/) (2023)
- [DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation](https://mlsanity.com/paper/2308.02915/) (2023)
- [Controllable Group Choreography using Contrastive Diffusion](https://mlsanity.com/paper/2310.18986/) (2023)
- [Dance Any Beat: Blending Beats with Visuals in Dance Video Generation](https://mlsanity.com/paper/2405.09266/) (2024)
- [LongDanceDiff: Long-term Dance Generation with Conditional Diffusion Model](https://mlsanity.com/paper/2308.11945/) (2023)
- [A Survey of Deep Learning in Sports Applications: Perception, Comprehension, and Decision](https://mlsanity.com/paper/2307.03353/) (2023)
- [MusicFace: Music-driven Expressive Singing Face Synthesis](https://mlsanity.com/paper/2303.14044/) (2023)
- [PianoMotion10M: Dataset and Benchmark for Hand Motion Generation in Piano Performance](https://mlsanity.com/paper/2406.09326/) (2024)
- [Scalable Group Choreography via Variational Phase Manifold Learning](https://mlsanity.com/paper/2407.18839/) (2024)
- [TCDiff++: An End-to-end Trajectory-Controllable Diffusion Model for Harmonious Music-Driven Group Choreography](https://mlsanity.com/paper/2506.18671/) (2025)
- [ChoreoMuse: Robust Music-to-Dance Video Generation with Style Transfer and Beat-Adherent Motion](https://mlsanity.com/paper/2507.19836/) (2025)
- [Harmonious Group Choreography with Trajectory-Controllable Diffusion](https://mlsanity.com/paper/2403.06189/) (2024)
- [Dance Generation by Sound Symbolic Words](https://mlsanity.com/paper/2306.03646/) (2023)
- [ST-GDance: Long-Term and Collision-Free Group Choreography from Music](https://mlsanity.com/paper/2507.21518/) (2025)
- [Global Position Aware Group Choreography using Large Language Model](https://mlsanity.com/paper/2503.09645/) (2025)
- [Listen to Rhythm, Choose Movements: Autoregressive Multimodal Dance Generation via Diffusion and Mamba with Decoupled Dance Dataset](https://mlsanity.com/paper/2601.03323/) (2026)
- [Skeleton2Stage: Reward-Guided Fine-Tuning for Physically Plausible Dance Generation](https://mlsanity.com/paper/2602.13778/) (2026)
- [ST-GDance++: A Scalable Spatial-Temporal Diffusion for Long-Duration Group Choreography](https://mlsanity.com/paper/2603.22316/) (2026)

## Abstract (arXiv)

Synthesizing human's movements such as dancing is a flourishing research field which has several applications in computer graphics. Recent studies have demonstrated the advantages of deep neural networks (DNNs) for achieving remarkable performance in motion and music tasks with little effort for feature pre-processing. However, applying DNNs for generating dance to a piece of music is nevertheless challenging, because of 1) DNNs need to generate large sequences while mapping the music input, 2) the DNN needs to constraint the motion beat to the music, and 3) DNNs require a considerable amount of hand-crafted data. In this study, we propose a weakly supervised deep recurrent method for real-time basic dance generation with audio power spectrum as input. The proposed model employs convolutional layers and a multilayered Long Short-Term memory (LSTM) to process the audio input. Then, another deep LSTM layer decodes the target dance sequence. Notably, this end-to-end approach has 1) an auto-conditioned decode configuration that reduces accumulation of feedback error of large dance sequence, 2) uses a contrastive cost function to regulate the mapping between the music and motion beat, and 3) trains with weak labels generated from the motion beat, reducing the amount of hand-crafted data. We evaluate the proposed network based on i) the similarities between generated and the baseline dancer motion with a cross entropy measure for large dance sequences, and ii) accurate timing between the music and motion beat with an F-measure. Experimental results revealed that, after training using a small dataset, the model generates basic dance steps with low cross entropy and maintains an F-measure score similar to that of a baseline dancer.