# TeMuDance: Contrastive Alignment-Based Textual Control for Music-Driven Dance Generation

arXiv: https://arxiv.org/abs/2604.17005 · HTML: https://mlsanity.com/paper/2604.17005/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 18 Apr 2026.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [EDGE: Editable Dance Generation From Music](https://mlsanity.com/paper/2211.10658/) (2022)
- [Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory](https://mlsanity.com/paper/2203.13055/) (2022)
- [Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models](https://mlsanity.com/paper/2208.06677/) (2022)
- [A Spatio-temporal Transformer for 3D Human Motion Prediction](https://mlsanity.com/paper/2004.08692/) (2020)
- [Convolutional Sequence to Sequence Model for Human Dynamics](https://mlsanity.com/paper/1805.00655/) (2018)
- [DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformer](https://mlsanity.com/paper/2103.10206/) (2021)
- [Deep representation learning for human motion prediction and classification](https://mlsanity.com/paper/1702.07486/) (2017)
- [FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation](https://mlsanity.com/paper/2212.03741/) (2022)
- [Learning to Generate Diverse Dance Motions with Transformer](https://mlsanity.com/paper/2008.08171/) (2020)
- [Dance Revolution: Long-Term Dance Generation with Music via Curriculum Learning](https://mlsanity.com/paper/2006.06119/) (2020)
- [TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integration](https://mlsanity.com/paper/2304.02419/) (2023)
- [MotionFix: Text-Driven 3D Human Motion Editing](https://mlsanity.com/paper/2408.00712/) (2024)
- [Action-Agnostic Human Pose Forecasting](https://mlsanity.com/paper/1810.09676/) (2018)
- [Motion Anything: Any to Motion Generation](https://mlsanity.com/paper/2503.06955/) (2025)
- [A Survey on Deep Learning for Skeleton-Based Human Animation](https://mlsanity.com/paper/2110.06901/) (2021)
- [POPDG: Popular 3D Dance Generation with PopDanceSet](https://mlsanity.com/paper/2405.03178/) (2024)
- [DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation](https://mlsanity.com/paper/2308.02915/) (2023)
- [UniMuMo: Unified Text, Music and Motion Generation](https://mlsanity.com/paper/2410.04534/) (2024)
- [Bio-LSTM: A Biomechanically Inspired Recurrent Neural Network for 3D Pedestrian Pose and Gait Prediction](https://mlsanity.com/paper/1809.03705/) (2018)
- [Music2Dance: DanceNet for Music-driven Dance Generation](https://mlsanity.com/paper/2002.03761/) (2020)
- [DanceEditor: Towards Iterative Editable Music-driven Dance Generation with Open-Vocabulary Descriptions](https://mlsanity.com/paper/2508.17342/) (2025)
- [DanceChat: Large Language Model-Guided Music-to-Dance Generation](https://mlsanity.com/paper/2506.10574/) (2025)

## Abstract (arXiv)

Existing music-driven dance generation approaches have achieved strong realism and effective audio-motion alignment. However, they generally lack semantic controllability, making it difficult to guide specific movements through natural language descriptions. This limitation primarily stems from the absence of large-scale datasets that jointly align music, text, and motion for supervised learning of text-conditioned control. To address this challenge, we propose TeMuDance, a framework that enables text-based control for music-conditioned dance generation without requiring any manually annotated music-text-motion triplet dataset. TeMuDance introduces a motion-centred bridging paradigm that leverages motion as a shared semantic anchor to align disjoint music-dance and text-motion datasets within a unified embedding space, enabling cross-modal retrieval of missing modalities for end-to-end training. A lightweight text control branch is then trained on top of a frozen music-to-dance diffusion backbone, preserving rhythmic fidelity while enabling fine-grained semantic guidance. To further suppress noise inherent in the retrieved supervision, we design a dual-stream fine-tuning strategy with confidence-based filtering. We also propose a novel task-aligned metric that quantifies whether textual prompts induce the intended kinematic attributes under music conditioning. Extensive experiments demonstrate that TeMuDance achieves competitive dance quality while substantially improving text-conditioned control over existing methods.