# From Rigging to Waving: 3D-Guided Diffusion for Natural Animation of Hand-Drawn Characters

arXiv: https://arxiv.org/abs/2509.06573 · HTML: https://mlsanity.com/paper/2509.06573/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 8 Sep 2025.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [VideoComposer: Compositional Video Synthesis with Motion Controllability](https://mlsanity.com/paper/2306.02018/) (2023)
- [Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance](https://mlsanity.com/paper/2403.14781/) (2024)
- [CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model](https://mlsanity.com/paper/2403.05034/) (2024)
- [ProPainter: Improving Propagation and Transformer for Video Inpainting](https://mlsanity.com/paper/2309.03897/) (2023)
- [DwNet: Dense warp-based network for pose-guided human video generation](https://mlsanity.com/paper/1910.09139/) (2019)
- [Animate-X: Universal Character Image Animation with Enhanced Motion Representation](https://mlsanity.com/paper/2410.10306/) (2024)
- [ToonCrafter: Generative Cartoon Interpolation](https://mlsanity.com/paper/2405.17933/) (2024)
- [Photo Wake-Up: 3D Character Animation from a Single Photo](https://mlsanity.com/paper/1812.02246/) (2018)
- [A Recipe for Scaling up Text-to-Video Generation with Text-free Videos](https://mlsanity.com/paper/2312.15770/) (2023)
- [PhysAnimator: Physics-Guided Generative Cartoon Animation](https://mlsanity.com/paper/2501.16550/) (2025)
- [Light-A-Video: Training-free Video Relighting via Progressive Light Fusion](https://mlsanity.com/paper/2502.08590/) (2025)
- [DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance](https://mlsanity.com/paper/2406.18459/) (2024)
- [A Method for Animating Children's Drawings of the Human Figure](https://mlsanity.com/paper/2303.12741/) (2023)
- [Fast Complementary Dynamics via Skinning Eigenmodes](https://mlsanity.com/paper/2303.11886/) (2023)
- [DrawingSpinUp: 3D Animation from Single Character Drawings](https://mlsanity.com/paper/2409.08615/) (2024)
- [Complementary Dynamics](https://mlsanity.com/paper/2009.02462/) (2020)
- [MikuDance: Animating Character Art with Mixed Motion Dynamics](https://mlsanity.com/paper/2411.08656/) (2024)

## Built on by (most cited first, 2 of 2)

- [See-through: Single-image Layer Decomposition for Anime Characters](https://mlsanity.com/paper/2602.03749/) (2026)
- [Sketch2Motion: Text-driven 2D Sketch to 3D Animation via Diffusion-guided Skeleton Optimization](https://mlsanity.com/paper/2605.28394/) (2026)

## Abstract (arXiv)

Hand-drawn character animation is a vibrant field in computer graphics, presenting challenges in achieving geometric consistency while conveying expressive motion. Traditional skeletal animation methods maintain geometric consistency but struggle with complex non-rigid elements like flowing hair and skirts, leading to unnatural deformation. Conversely, video diffusion models synthesize realistic dynamics but often create geometric distortions in stylized drawings due to domain gaps. This work proposes a hybrid animation system that combines skeletal animation and video diffusion. Initially, coarse images are generated from characters retargeted with skeletal animations for geometric guidance. These images are then enhanced in texture and secondary dynamics using video diffusion priors, framing this enhancement as an inpainting task. A domain-adapted diffusion model refines user-masked regions needing improvement, especially for secondary dynamics. To enhance motion realism further, we introduce a Secondary Dynamics Injection (SDI) strategy in the denoising process, incorporating features from a pre-trained diffusion model enriched with human motion priors. Additionally, to tackle unnatural deformations from low-poly single-mesh character modeling, we present a Hair Layering Modeling (HLM) technique that uses segmentation maps to separate hair from the body, allowing for more natural animation of long-haired characters. Extensive experiments show that our system outperforms state-of-the-art methods in both quantitative and qualitative evaluations.