mlsanityData to 14 Oct 2026

AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillation

arXiv 2312.03795 ↗First posted 6 Dec 2023cs.CVcited by 28

Not on the map. Why?

Abstract (arXiv)

Advances in 3D generation have facilitated sequential 3D model generation (a.k.a 4D generation), yet its application for animatable objects with large motion remains scarce. Our work proposes AnimatableDreamer, a text-to-4D generation framework capable of generating diverse categories of non-rigid objects on skeletons extracted from a monocular video. At its core, AnimatableDreamer is equipped with our novel optimization design dubbed Canonical Score Distillation (CSD), which lifts 2D diffusion for temporal consistent 4D generation. CSD, designed from a score gradient perspective, generates a canonical model with warp-robustness across different articulations. Notably, it also enhances the authenticity of bones and skinning by integrating inductive priors from a diffusion model. Furthermore, with multi-view distillation, CSD infers invisible regions, thereby improving the fidelity of monocular non-rigid reconstruction. Extensive experiments demonstrate the capability of our method in generating high-flexibility text-guided 3D models from the monocular video, while also showing improved reconstruction performance over existing non-rigid reconstruction methods.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 48 of its 58 references that are in the map. First posted on arXiv 6 Dec 2023.

Builds on

48 of 58 references.

References · 48

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  5. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  6. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  7. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  8. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  9. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  10. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  11. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  12. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
36 more
  1. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  2. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
  3. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  4. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 843
  5. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 751
  6. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 746
  7. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  8. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 670
  9. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  10. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  11. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 615
  12. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  13. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 587
  14. 2023Neuralangelo: High-Fidelity Neural Surface Reconstructioncited by 467
  15. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 460
  16. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  17. 2022NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstructioncited by 314
  18. 2020Neural Radiance Flow for 4D View Synthesis and Video Processingcited by 270
  19. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 260
  20. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  21. 2021SNARF: Differentiable Forward Skinning for Animating Non-Rigid Neural Implicit Shapescited by 221
  22. 2022Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Renderingcited by 217
  23. 2023Text-To-4D Dynamic Scene Generationcited by 205
  24. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 197
  25. 2022Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-timecited by 165
  26. 2022ARF: Artistic Radiance Fieldscited by 147
  27. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 112
  28. 2023DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Renderingcited by 103
  29. 2023SceneDreamer: Unbounded 3D Scene Generation from 2D Image Collectionscited by 72
  30. 2021Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objectscited by 50
  31. 2023Reconstructing Animatable Categories from Videoscited by 47
  32. 2021Learning to Segment Rigid Motions from Two Framescited by 46
  33. 2023Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusioncited by 38
  34. 2023ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collectionscited by 22
  35. 2023Root Pose Decomposition Towards Generic Non-rigid 3D Reconstruction with Monocular Videoscited by 8
  36. 2023Learning Articulated Shape with Keypoint Pseudo-labels from Web Imagescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 14; 2025: 11; 2026: 3.

28 papers.

Sort:

By area:

  1. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 215
  2. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusioncited by 140
  3. 2024STAG4D: Spatial-Temporal Anchored Generative 4D Gaussianscited by 102
  4. 2024V3D: Video Diffusion Models are Effective 3D Generatorscited by 89
  5. 2025DeepMesh: Auto-Regressive Artist-mesh Creation with Reinforcement Learningcited by 65
  6. 2024SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfercited by 61
  7. 2025Puppeteer: Rig and Animate Your 3D Modelscited by 43
  8. 2024Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfelscited by 38
  9. 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understandingcited by 38
  10. 2025NANO3D: A Training-Free Approach for Efficient 3D Editing Without Maskscited by 31
  11. 2024AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animationcited by 25
  12. 2024VividDreamer: Invariant Score Distillation For Hyper-Realistic Text-to-3D Generationcited by 15
16 more
  1. 20244Dynamic: Text-to-4D Generation with Hybrid Priorscited by 15
  2. 2024Isotropic3D: Image-to-3D Generation Based on a Single CLIP Embeddingcited by 13
  3. 2024FlexiDreamer: Single Image-to-3D Generation with FlexiCubescited by 12
  4. 2025MeshRipple: Structured Autoregressive Generation of Artist-Meshescited by 10
  5. 2025CoRe3D: Collaborative Reasoning as a Foundation for 3D Intelligencecited by 10
  6. 2025Splat4D: Diffusion-Enhanced 4D Gaussian Splatting for Temporally and Spatially Consistent Content Creationcited by 9
  7. 2026PhysForge: Generating Physics-Grounded 3D Assets for Interactive Virtual Worldcited by 9
  8. 2025PAD3R: Pose-Aware Dynamic 3D Reconstruction from Casual Videoscited by 7
  9. 2025Video4DGen: Enhancing Video and 4D Generation through Mutual Optimizationcited by 5
  10. 2026When Policy Entropy Constraint Fails: Preserving Diversity in Flow-based RLHF via Perceptual Entropycited by 3
  11. 2025Gaussian See, Gaussian Do: Semantic 3D Motion Transfer from Multiview Videocited by 1
  12. 2026SK-Adapter: Skeleton-Based Structural Control for Native 3D Generationcited by 1
  13. 2024MotionDreamer: Exploring Semantic Video Diffusion features for Zero-Shot 3D Mesh Animationcited by 0
  14. 2024Hierarchically Structured Neural Bones for Reconstructing Animatable Objects from Casual Videoscited by 0
  15. 2024Focus on Neighbors and Know the Whole: Towards Consistent Dense Multiview Text-to-Image Generator for 3D Creationcited by 0
  16. 2025AnchorHOI: Zero-shot Generation of 4D Human-Object Interaction via Anchor-based Prior Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AniGen: Unified S3 Fields for Animatable 3D Asset Generation
  2. 2025AnimaMimic: Imitating 3D Animation from Video Priors
  3. 2025TriDiff-4D: Fast 4D Generation through Diffusion-based Triplane Re-posing
  4. 2023Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models
  5. 2025Articulated Kinematics Distillation from Video Diffusion Models
  6. 2024CT4D: Consistent Text-to-4D Generation with Animatable Meshes
  7. 2025AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion Models
  8. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusion