mlsanityData to 14 Oct 2026

PF-D2M: A Pose-free Diffusion Model for Universal Dance-to-Music Generation

arXiv 2601.15872 ↗First posted 22 Jan 2026cs.SD cs.CV cs.LGcited by 0

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Dance-to-music generation aims to generate music that is aligned with dance movements. Existing approaches typically rely on body motion features extracted from a single human dancer and limited dance-to-music datasets, which restrict their performance and applicability to real-world scenarios involving multiple dancers and non-human dancers. In this paper, we propose PF-D2M, a universal diffusion-based dance-to-music generation model that incorporates visual features extracted from dance videos. PF-D2M is trained with a progressive training strategy that effectively addresses data scarcity and generalization challenges. Both objective and subjective evaluations show that PF-D2M achieves state-of-the-art performance in dance-music alignment and music quality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 24 of its 33 references that are in the map. First posted on arXiv 22 Jan 2026.

Builds on

24 of 33 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation15 Jun

Other references · 23

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2022Classifier-Free Diffusion Guidancecited by 5,847
  4. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  5. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  6. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 938
  7. 2020VGGSound: A Large-scale Audio-Visual Datasetcited by 648
  8. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
  9. 2024Qwen2-Audio Technical Reportcited by 560
  10. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
  11. 2016FMA: A Dataset For Music Analysiscited by 207
  12. 2024Stable Audio Opencited by 197
11 more
  1. 2022Hybrid Transformers for Music Source Separationcited by 134
  2. 2024MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesiscited by 134
  3. 2024Synchformer: Efficient Synchronization from Sparse Cuescited by 97
  4. 2025YuE: Scaling Open Foundation Models for Long-Form Music Generationcited by 71
  5. 2024Video-Guided Foley Sound Generation with Multimodal Controlscited by 49
  6. 2023Moisesdb: A dataset for source separation beyond 4-stemscited by 39
  7. 2022Quantized GAN for Complex Music Generation from Dance Videoscited by 37
  8. 2023Long-Term Rhythmic Video Soundtrackercited by 17
  9. 2021Dance2Music: Automatic Dance-driven Music Generationcited by 16
  10. 2024Dance-to-Music Generation with Encoder-based Textual Inversioncited by 12
  11. 2025SAGA-SR: Semantically and Acoustically Guided Audio Super-Resolutioncited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Dance Any Beat: Blending Beats with Visuals in Dance Video Generation
  2. 2026Dance to Music Generation leveraging Pre-training with Unpaired data and Contrastive Alignment ↗
  3. 2025X-Dancer: Expressive Music to Human Dance Video Generation
  4. 2023DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation
  5. 2024Bidirectional Autoregressive Diffusion Model for Dance Generation
  6. 2023DanceMeld: Unraveling Dance Phrases with Hierarchical Latent Codes for Music-to-Dance Synthesis
  7. 2026DanceDuo: Bridging Human Movement and AI Choreography
  8. 2025MotionRAG-Diff: A Retrieval-Augmented Diffusion Framework for Long-Term Music-to-Dance Generation