mlsanityData to 14 Oct 2026

TeMuDance: Contrastive Alignment-Based Textual Control for Music-Driven Dance Generation

arXiv 2604.17005 ↗First posted 18 Apr 2026cs.CV cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Existing music-driven dance generation approaches have achieved strong realism and effective audio-motion alignment. However, they generally lack semantic controllability, making it difficult to guide specific movements through natural language descriptions. This limitation primarily stems from the absence of large-scale datasets that jointly align music, text, and motion for supervised learning of text-conditioned control. To address this challenge, we propose TeMuDance, a framework that enables text-based control for music-conditioned dance generation without requiring any manually annotated music-text-motion triplet dataset. TeMuDance introduces a motion-centred bridging paradigm that leverages motion as a shared semantic anchor to align disjoint music-dance and text-motion datasets within a unified embedding space, enabling cross-modal retrieval of missing modalities for end-to-end training. A lightweight text control branch is then trained on top of a frozen music-to-dance diffusion backbone, preserving rhythmic fidelity while enabling fine-grained semantic guidance. To further suppress noise inherent in the retrieved supervision, we design a dual-stream fine-tuning strategy with confidence-based filtering. We also propose a novel task-aligned metric that quantifies whether textual prompts induce the intended kinematic attributes under music conditioning. Extensive experiments demonstrate that TeMuDance achieves competitive dance quality while substantially improving text-conditioned control over existing methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 49 references that are in the map. First posted on arXiv 18 Apr 2026.

Builds on

25 of 49 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  3. 2017On human motion prediction using recurrent neural networkscited by 338
  4. 2022EDGE: Editable Dance Generation From Musiccited by 287
  5. 2022Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memorycited by 212
  6. 2022Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Modelscited by 174
  7. 2020A Spatio-temporal Transformer for 3D Human Motion Predictioncited by 133
  8. 2018Convolutional Sequence to Sequence Model for Human Dynamicscited by 115
  9. 2021DanceFormer: Music Conditioned 3D Dance Generation with Parametric Motion Transformercited by 111
  10. 2017Deep representation learning for human motion prediction and classificationcited by 98
  11. 2022FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generationcited by 88
  12. 2020Learning to Generate Diverse Dance Motions with Transformercited by 85
13 more
  1. 2020Dance Revolution: Long-Term Dance Generation with Music via Curriculum Learningcited by 81
  2. 2023TM2D: Bimodality Driven 3D Dance Generation via Music-Text Integrationcited by 76
  3. 2024MotionFix: Text-Driven 3D Human Motion Editingcited by 64
  4. 2018Action-Agnostic Human Pose Forecastingcited by 56
  5. 2025Motion Anything: Any to Motion Generationcited by 42
  6. 2021A Survey on Deep Learning for Skeleton-Based Human Animationcited by 27
  7. 2024POPDG: Popular 3D Dance Generation with PopDanceSetcited by 24
  8. 2023DiffDance: Cascaded Human Motion Diffusion Model for Dance Generationcited by 23
  9. 2024UniMuMo: Unified Text, Music and Motion Generationcited by 19
  10. 2018Bio-LSTM: A Biomechanically Inspired Recurrent Neural Network for 3D Pedestrian Pose and Gait Predictioncited by 13
  11. 2020Music2Dance: DanceNet for Music-driven Dance Generationcited by 6
  12. 2025DanceEditor: Towards Iterative Editable Music-driven Dance Generation with Open-Vocabulary Descriptionscited by 6
  13. 2025DanceChat: Large Language Model-Guided Music-to-Dance Generationcited by 5

Built on by

Not cited yet.