Music- and Lyrics-driven Dance Synthesis
Not on the map. Why?
Abstract (arXiv)
Lyrics often convey information about the songs that are beyond the auditory dimension, enriching the semantic meaning of movements and musical themes. Such insights are important in the dance choreography domain. However, most existing dance synthesis methods mainly focus on music-to-dance generation, without considering the semantic information. To complement it, we introduce JustLMD, a new multimodal dataset of 3D dance motion with music and lyrics. To the best of our knowledge, this is the first dataset with triplet information including dance motion, music, and lyrics. Additionally, we showcase a cross-modal diffusion-based network designed to generate 3D dance motion conditioned on music and lyrics. The proposed JustLMD dataset encompasses 4.6 hours of 3D dance motion in 1867 sequences, accompanied by musical tracks and their corresponding English lyrics.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 16 references that are in the map. First posted on arXiv 30 Sep 2023.
Builds on
9 of 16 references.
References · 9
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,323
- 2020Jukebox: A Generative Model for Musiccited by 542
- 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
- 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 190
- 2020Motion Capture from Internet Videoscited by 33
Built on by
Not cited yet.