# BiTDiff: Fine-Grained 3D Conducting Motion Generation via BiMamba-Transformer Diffusion arXiv: https://arxiv.org/abs/2604.04395 ยท HTML: https://mlsanity.com/paper/2604.04395/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 41 references that are in the map. - First posted on arXiv 6 Apr 2026. ## Builds on (most cited first, 30 of 30) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023) - [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017) - [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019) - [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019) - [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018) - [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021) - [EDGE: Editable Dance Generation From Music](https://mlsanity.com/paper/2211.10658/) (2022) - [Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory](https://mlsanity.com/paper/2203.13055/) (2022) - [Generating Holistic 3D Human Motion from Speech](https://mlsanity.com/paper/2212.04420/) (2022) - [EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling](https://mlsanity.com/paper/2401.00374/) (2023) - [FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generation](https://mlsanity.com/paper/2212.03741/) (2022) - [SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces](https://mlsanity.com/paper/2306.10799/) (2023) - [DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generation](https://mlsanity.com/paper/2401.04747/) (2024) - [Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitives](https://mlsanity.com/paper/2403.10518/) (2024) - [Visual Speech-Aware Perceptual 3D Facial Expression Reconstruction from Videos](https://mlsanity.com/paper/2207.11094/) (2022) - [GestureLSM: Latent Shortcut based Co-Speech Gesture Generation with Spatial-Temporal Modeling](https://mlsanity.com/paper/2501.18898/) (2025) - [SemTalk: Holistic Co-speech Motion Generation with Frame-level Semantic Emphasis](https://mlsanity.com/paper/2412.16563/) (2024) - [Lodge++: High-quality and Long Dance Generation with Vivid Choreography Patterns](https://mlsanity.com/paper/2410.20389/) (2024) - [Predicting 4D Hand Trajectory from Monocular Videos](https://mlsanity.com/paper/2501.08329/) (2025) - [Freetalker: Controllable Speech and Text-Driven Gesture Generation Based on Diffusion Models for Enhanced Speaker Naturalness](https://mlsanity.com/paper/2401.03476/) (2024) - [MEGADance: Mixture-of-Experts Architecture for Genre-Aware 3D Dance Generation](https://mlsanity.com/paper/2505.17543/) (2025) - [EchoMask: Speech-Queried Attention-based Mask Modeling for Holistic Co-Speech Motion Generation](https://mlsanity.com/paper/2504.09209/) (2025) - [BeatDance: A Beat-Based Model-Agnostic Contrastive Learning Framework for Music-Dance Retrieval](https://mlsanity.com/paper/2310.10300/) (2023) - [Mitigating Error Accumulation in Co-Speech Motion Generation via Global Rotation Diffusion and Multi-Level Constraints](https://mlsanity.com/paper/2511.10076/) (2025) - [FlowerDance: MeanFlow for Efficient and Refined 3D Dance Generation](https://mlsanity.com/paper/2511.21029/) (2025) - [CoheDancers: Enhancing Interactive Group Dance Generation through Music-Driven Coherence Decomposition](https://mlsanity.com/paper/2412.19123/) (2024) - [Taming Diffusion Models for Music-driven Conducting Motion Generation](https://mlsanity.com/paper/2306.10065/) (2023) - [Music to Dance as Language Translation using Sequence Models](https://mlsanity.com/paper/2403.15569/) (2024) - [TokenDance: Token-to-Token Music-to-Dance Generation with Bidirectional Mamba](https://mlsanity.com/paper/2603.27314/) (2026) ## Built on by (most cited first, 2 of 2) - [OmniDance: Multimodal Driven Dance Video Generation with Large-scale Internet Data](https://mlsanity.com/paper/2606.30019/) (2026) - [Interactive Multi-Turn Retrieval for Health Videos](https://mlsanity.com/paper/2605.01409/) (2026) ## Abstract (arXiv) 3D conducting motion generation aims to synthesize fine-grained conductor motions from music, with broad potential in music education, virtual performance, digital human animation, and human-AI co-creation. However, this task remains underexplored due to two major challenges: (1) the lack of large-scale fine-grained 3D conducting datasets and (2) the absence of effective methods that can jointly support long-sequence generation with high quality and efficiency. To address the data limitation, we develop a quality-oriented 3D conducting motion collection pipeline and construct CM-Data, a fine-grained SMPL-X dataset with about 10 hours of conducting motion data. To the best of our knowledge, CM-Data is the first and largest public dataset for 3D conducting motion generation. To address the methodological limitation, we propose BiTDiff, a novel framework for 3D conducting motion generation, built upon a BiMamba-Transformer hybrid model architecture for efficient long-sequence modeling and a Diffusion-based generative strategy with human-kinematic decomposition for high-quality motion synthesis. Specifically, BiTDiff introduces auxiliary physical-consistency losses and a hand-/body-specific forward-kinematics design for better fine-grained motion modeling, while leveraging BiMamba for memory-efficient long-sequence temporal modeling and Transformer for cross-modal semantic alignment. In addition, BiTDiff supports training-free joint-level motion editing, enabling downstream human-AI interaction design. Extensive quantitative and qualitative experiments demonstrate that BiTDiff achieves state-of-the-art (SOTA) performance for 3D conducting motion generation on the CM-Data dataset. Code will be available upon acceptance.