# LongDanceDiff: Long-term Dance Generation with Conditional Diffusion Model

arXiv: https://arxiv.org/abs/2308.11945 · HTML: https://mlsanity.com/paper/2308.11945/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 23 Aug 2023.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models](https://mlsanity.com/paper/2210.08933/) (2022)
- [EDGE: Editable Dance Generation From Music](https://mlsanity.com/paper/2211.10658/) (2022)
- [Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory](https://mlsanity.com/paper/2203.13055/) (2022)
- [Symbolic Music Generation with Diffusion Models](https://mlsanity.com/paper/2103.16091/) (2021)
- [DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformer](https://mlsanity.com/paper/2103.10206/) (2021)
- [Learning to Generate Diverse Dance Motions with Transformer](https://mlsanity.com/paper/2008.08171/) (2020)
- [Dance Revolution: Long-Term Dance Generation with Music via Curriculum Learning](https://mlsanity.com/paper/2006.06119/) (2020)
- [Mousai: Text-to-Music Generation with Long-Context Latent Diffusion](https://mlsanity.com/paper/2301.11757/) (2023)
- [SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers](https://mlsanity.com/paper/2212.10325/) (2022)
- [Farewell to Mutual Information: Variational Distillation for Cross-Modal Person Re-Identification](https://mlsanity.com/paper/2104.02862/) (2021)
- [Multi-instrument Music Synthesis with Spectrogram Diffusion](https://mlsanity.com/paper/2206.05408/) (2022)
- [Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Model](https://mlsanity.com/paper/2210.12315/) (2022)
- [Learning to dance: A graph convolutional adversarial network to generate realistic dance motions from audio](https://mlsanity.com/paper/2011.12999/) (2020)
- [Weakly Supervised Deep Recurrent Neural Networks for Basic Dance Step Generation](https://mlsanity.com/paper/1807.01126/) (2018)
- [Modiff: Action-Conditioned 3D Motion Generation with Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2301.03949/) (2023)

## Built on by (most cited first, 12 of 12)

- [CG-HOI: Contact-Guided 3D Human-Object Interaction Generation](https://mlsanity.com/paper/2311.16097/) (2023)
- [Large Motion Model for Unified Multi-Modal Motion Generation](https://mlsanity.com/paper/2404.01284/) (2024)
- [Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitives](https://mlsanity.com/paper/2403.10518/) (2024)
- [CustomListener: Text-guided Responsive Interaction for User-friendly Listening Head Generation](https://mlsanity.com/paper/2403.00274/) (2024)
- [Lodge++: High-quality and Long Dance Generation with Vivid Choreography Patterns](https://mlsanity.com/paper/2410.20389/) (2024)
- [Lagrangian Motion Fields for Long-term Motion Generation](https://mlsanity.com/paper/2409.01522/) (2024)
- [DanceChat: Large Language Model-Guided Music-to-Dance Generation](https://mlsanity.com/paper/2506.10574/) (2025)
- [TCDiff++: An End-to-end Trajectory-Controllable Diffusion Model for Harmonious Music-Driven Group Choreography](https://mlsanity.com/paper/2506.18671/) (2025)
- [Harmonious Group Choreography with Trajectory-Controllable Diffusion](https://mlsanity.com/paper/2403.06189/) (2024)
- [Automatic Generation of Aerobatic Flight in Complex Environments via Diffusion Models](https://mlsanity.com/paper/2504.15138/) (2025)
- [GCDance: Genre-Controlled Music-Driven 3D Full Body Dance Generation](https://mlsanity.com/paper/2502.18309/) (2025)
- [DanceDuo: Bridging Human Movement and AI Choreography](https://mlsanity.com/paper/2606.26507/) (2026)

## Abstract (arXiv)

Dancing with music is always an essential human art form to express emotion. Due to the high temporal-spacial complexity, long-term 3D realist dance generation synchronized with music is challenging. Existing methods suffer from the freezing problem when generating long-term dances due to error accumulation and training-inference discrepancy. To address this, we design a conditional diffusion model, LongDanceDiff, for this sequence-to-sequence long-term dance generation, addressing the challenges of temporal coherency and spatial constraint. LongDanceDiff contains a transformer-based diffusion model, where the input is a concatenation of music, past motions, and noised future motions. This partial noising strategy leverages the full-attention mechanism and learns the dependencies among music and past motions. To enhance the diversity of generated dance motions and mitigate the freezing problem, we introduce a mutual information minimization objective that regularizes the dependency between past and future motions. We also address common visual quality issues in dance generation, such as foot sliding and unsmooth motion, by incorporating spatial constraints through a Global-Trajectory Modulation (GTM) layer and motion perceptual losses, thereby improving the smoothness and naturalness of motion generation. Extensive experiments demonstrate a significant improvement in our approach over the existing state-of-the-art methods. We plan to release our codes and models soon.