# AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillation

arXiv: https://arxiv.org/abs/2312.03795 · HTML: https://mlsanity.com/paper/2312.03795/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 58 references that are in the map.
- First posted on arXiv 6 Dec 2023.

## Builds on (most cited first, 30 of 48)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [4D Gaussian Splatting for Real-Time Dynamic Scene Rendering](https://mlsanity.com/paper/2310.08528/) (2023)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators](https://mlsanity.com/paper/2303.13439/) (2023)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputs](https://mlsanity.com/paper/2112.00724/) (2021)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [Neuralangelo: High-Fidelity Neural Surface Reconstruction](https://mlsanity.com/paper/2306.03092/) (2023)
- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction](https://mlsanity.com/paper/2212.05231/) (2022)
- [Neural Radiance Flow for 4D View Synthesis and Video Processing](https://mlsanity.com/paper/2012.09790/) (2020)

## Built on by (most cited first, 28 of 28)

- [CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model](https://mlsanity.com/paper/2403.05034/) (2024)
- [DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion](https://mlsanity.com/paper/2411.04928/) (2024)
- [STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians](https://mlsanity.com/paper/2403.14939/) (2024)
- [V3D: Video Diffusion Models are Effective 3D Generators](https://mlsanity.com/paper/2403.06738/) (2024)
- [DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learning](https://mlsanity.com/paper/2503.15265/) (2025)
- [SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer](https://mlsanity.com/paper/2404.03736/) (2024)
- [Puppeteer: Rig and Animate Your 3D Models](https://mlsanity.com/paper/2508.10898/) (2025)
- [Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfels](https://mlsanity.com/paper/2405.16822/) (2024)
- [ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understanding](https://mlsanity.com/paper/2506.01853/) (2025)
- [NANO3D: A Training-Free Approach for Efficient 3D Editing Without Masks](https://mlsanity.com/paper/2510.15019/) (2025)
- [AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animation](https://mlsanity.com/paper/2410.07164/) (2024)
- [VividDreamer: Invariant Score Distillation For Hyper-Realistic Text-to-3D Generation](https://mlsanity.com/paper/2407.09822/) (2024)
- [4Dynamic: Text-to-4D Generation with Hybrid Priors](https://mlsanity.com/paper/2407.12684/) (2024)
- [Isotropic3D: Image-to-3D Generation Based on a Single CLIP Embedding](https://mlsanity.com/paper/2403.10395/) (2024)
- [FlexiDreamer: Single Image-to-3D Generation with FlexiCubes](https://mlsanity.com/paper/2404.00987/) (2024)
- [MeshRipple: Structured Autoregressive Generation of Artist-Meshes](https://mlsanity.com/paper/2512.07514/) (2025)
- [CoRe3D: Collaborative Reasoning as a Foundation for 3D Intelligence](https://mlsanity.com/paper/2512.12768/) (2025)
- [Splat4D: Diffusion-Enhanced 4D Gaussian Splatting for Temporally and Spatially Consistent Content Creation](https://mlsanity.com/paper/2508.07557/) (2025)
- [PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual World](https://mlsanity.com/paper/2605.05163/) (2026)
- [PAD3R: Pose-Aware Dynamic 3D Reconstruction from Casual Videos](https://mlsanity.com/paper/2509.25183/) (2025)
- [Video4DGen: Enhancing Video and 4D Generation through Mutual Optimization](https://mlsanity.com/paper/2504.04153/) (2025)
- [When Policy Entropy Constraint Fails: Preserving Diversity in Flow-based RLHF via Perceptual Entropy](https://mlsanity.com/paper/2605.12112/) (2026)
- [Gaussian See, Gaussian Do: Semantic 3D Motion Transfer from Multiview Video](https://mlsanity.com/paper/2511.14848/) (2025)
- [SK-Adapter: Skeleton-Based Structural Control for Native 3D Generation](https://mlsanity.com/paper/2603.14152/) (2026)
- [MotionDreamer: Exploring Semantic Video Diffusion features for Zero-Shot 3D Mesh Animation](https://mlsanity.com/paper/2405.20155/) (2024)
- [Hierarchically Structured Neural Bones for Reconstructing Animatable Objects from Casual Videos](https://mlsanity.com/paper/2408.00351/) (2024)
- [Focus on Neighbors and Know the Whole: Towards Consistent Dense Multiview Text-to-Image Generator for 3D Creation](https://mlsanity.com/paper/2408.13149/) (2024)
- [AnchorHOI: Zero-shot Generation of 4D Human-Object Interaction via Anchor-based Prior Distillation](https://mlsanity.com/paper/2512.14095/) (2025)

## Abstract (arXiv)

Advances in 3D generation have facilitated sequential 3D model generation (a.k.a 4D generation), yet its application for animatable objects with large motion remains scarce. Our work proposes AnimatableDreamer, a text-to-4D generation framework capable of generating diverse categories of non-rigid objects on skeletons extracted from a monocular video. At its core, AnimatableDreamer is equipped with our novel optimization design dubbed Canonical Score Distillation (CSD), which lifts 2D diffusion for temporal consistent 4D generation. CSD, designed from a score gradient perspective, generates a canonical model with warp-robustness across different articulations. Notably, it also enhances the authenticity of bones and skinning by integrating inductive priors from a diffusion model. Furthermore, with multi-view distillation, CSD infers invisible regions, thereby improving the fidelity of monocular non-rigid reconstruction. Extensive experiments demonstrate the capability of our method in generating high-flexibility text-guided 3D models from the monocular video, while also showing improved reconstruction performance over existing non-rigid reconstruction methods.