# DisCo: Disentangled Control for Realistic Human Dance Generation

arXiv: https://arxiv.org/abs/2307.00040 · HTML: https://mlsanity.com/paper/2307.00040/

## Key facts

- Cited by 142 papers in the map (data through 14 Oct 2026).
- Builds on 69 of its 80 references that are in the map.
- First posted on arXiv 30 Jun 2023.

## Builds on (most cited first, 30 of 69)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)

## Built on by (most cited first, 30 of 142)

- [Multimodal Foundation Models: From Specialists to General-Purpose Assistants](https://mlsanity.com/paper/2309.10020/) (2023)
- [LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control](https://mlsanity.com/paper/2407.03168/) (2024)
- [MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance](https://mlsanity.com/paper/2406.19680/) (2024)
- [Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation](https://mlsanity.com/paper/2406.01900/) (2024)
- [Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motion](https://mlsanity.com/paper/2402.03162/) (2024)
- [ControlNeXt: Powerful and Efficient Control for Image and Video Generation](https://mlsanity.com/paper/2408.06070/) (2024)
- [OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models](https://mlsanity.com/paper/2502.01061/) (2025)
- [Controllable Video Generation: A Survey](https://mlsanity.com/paper/2507.16869/) (2025)
- [Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation](https://mlsanity.com/paper/2506.09350/) (2025)
- [StableAnimator: High-Quality Identity-Preserving Human Image Animation](https://mlsanity.com/paper/2411.17697/) (2024)
- [OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generation](https://mlsanity.com/paper/2412.00115/) (2024)
- [MIMO: Controllable Character Video Synthesis with Spatial Decomposed Modeling](https://mlsanity.com/paper/2409.16160/) (2024)
- [Diffusion Model-Based Video Editing: A Survey](https://mlsanity.com/paper/2407.07111/) (2024)
- [ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Models](https://mlsanity.com/paper/2506.16211/) (2025)
- [Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animation](https://mlsanity.com/paper/2509.16630/) (2025)
- [EnerVerse-AC: Envisioning Embodied Environments with Action Condition](https://mlsanity.com/paper/2505.09723/) (2025)
- [Motion Consistency Model: Accelerating Video Diffusion with Disentangled Motion-Appearance Distillation](https://mlsanity.com/paper/2406.06890/) (2024)
- [TrackGo: A Flexible and Efficient Method for Controllable Video Generation](https://mlsanity.com/paper/2408.11475/) (2024)
- [CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attention](https://mlsanity.com/paper/2409.01876/) (2024)
- [HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generation](https://mlsanity.com/paper/2502.04847/) (2025)
- [A Survey on Generative AI and LLM for Video Generation, Understanding, and Streaming](https://mlsanity.com/paper/2404.16038/) (2024)
- [RealisDance: Equip controllable character animation with realistic hands](https://mlsanity.com/paper/2409.06202/) (2024)
- [DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidance](https://mlsanity.com/paper/2504.01724/) (2025)
- [DisPose: Disentangling Pose Guidance for Controllable Human Image Animation](https://mlsanity.com/paper/2412.09349/) (2024)
- [Tunnel Try-on: Excavating Spatial-temporal Tunnels for High-quality Virtual Try-on in Videos](https://mlsanity.com/paper/2404.17571/) (2024)
- [RealisDance-DiT: Simple yet Strong Baseline towards Controllable Character Animation in the Wild](https://mlsanity.com/paper/2504.14977/) (2025)
- [MegActor: Harness the Power of Raw Video for Vivid Portrait Animation](https://mlsanity.com/paper/2405.20851/) (2024)
- [A Comprehensive Survey on Human Video Generation: Challenges, Methods, and Insights](https://mlsanity.com/paper/2407.08428/) (2024)
- [FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios](https://mlsanity.com/paper/2505.03730/) (2025)
- [Human Motion Video Generation: A Survey](https://mlsanity.com/paper/2509.03883/) (2025)

## Abstract (arXiv)

Generative AI has made significant strides in computer vision, particularly in text-driven image/video synthesis (T2I/T2V). Despite the notable advancements, it remains challenging in human-centric content synthesis such as realistic dance generation. Current methodologies, primarily tailored for human motion transfer, encounter difficulties when confronted with real-world dance scenarios (e.g., social media dance), which require to generalize across a wide spectrum of poses and intricate human details. In this paper, we depart from the traditional paradigm of human motion transfer and emphasize two additional critical attributes for the synthesis of human dance content in social media contexts: (i) Generalizability: the model should be able to generalize beyond generic human viewpoints as well as unseen human subjects, backgrounds, and poses; (ii) Compositionality: it should allow for the seamless composition of seen/unseen subjects, backgrounds, and poses from different sources. To address these challenges, we introduce DISCO, which includes a novel model architecture with disentangled control to improve the compositionality of dance synthesis, and an effective human attribute pre-training for better generalizability to unseen humans. Extensive qualitative and quantitative results demonstrate that DisCc can generate high-quality human dance images and videos with diverse appearances and flexible motions. Code is available at https://disco-dance.github.io/.