mlsanityData to 7 Oct 2026

Reconstructing Animatable Categories from Videos

arXiv 2305.06351 ↗First posted 10 May 2023cs.CV cs.GRcited by 46

Not on the map. Why?

Abstract (arXiv)

Building animatable 3D models is challenging due to the need for 3D scans, laborious registration, and manual rigging, which are difficult to scale to arbitrary categories. Recently, differentiable rendering provides a pathway to obtain high-quality 3D models from monocular videos, but these are limited to rigid categories or single instances. We present RAC that builds category 3D models from monocular videos while disentangling variations over instances and motion over time. Three key ideas are introduced to solve this problem: (1) specializing a skeleton to instances via optimization, (2) a method for latent space regularization that encourages shared structure across a category while maintaining instance details, and (3) using 3D background models to disentangle objects from the background. We show that 3D models of humans, cats, and dogs can be learned from 50-100 internet videos.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 62 of its 82 references that are in the map. First posted on arXiv 10 May 2023.

Builds on

62 of 82 references.

References · 62

  1. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  2. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  3. 2016Density estimation using Real NVPcited by 2,029
  4. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  5. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  6. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  7. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  8. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  9. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  10. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  11. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  12. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
38 more
  1. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  2. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  3. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  4. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  5. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  6. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  7. 2019PointRend: Image Segmentation as Renderingcited by 439
  8. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  9. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  10. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  11. 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 310
  12. 2021ICON: Implicit Clothed humans Obtained from Normalscited by 283
  13. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
  14. 2021A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Posecited by 233
  15. 2020PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstructioncited by 231
  16. 2021HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fieldscited by 210
  17. 2021Neural Articulated Radiance Fieldcited by 196
  18. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 195
  19. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  20. 2021SCANimate: Weakly Supervised Learning of Skinned Clothed Avatar Networkscited by 181
  21. 2022TAVA: Template-free Animatable Volumetric Actorscited by 172
  22. 2019Tex2Shape: Detailed Full Human Body Geometry From a Single Imagecited by 166
  23. 2018Monocular Total Capture: Posing Face, Body, and Hands in the Wildcited by 162
  24. 2021NeRF--: Neural Radiance Fields Without Known Camera Parameterscited by 155
  25. 2022SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Videocited by 144
  26. 2019From Variational to Deterministic Autoencoderscited by 131
  27. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  28. 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 115
  29. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 111
  30. 2019Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"cited by 106
  31. 2018Creatures great and SMAL: Recovering the shape and motion of animals from videocited by 84
  32. 2020Who Left the Dogs Out? 3D Animal Reconstruction with Expectation Maximization in the Loopcited by 81
  33. 2020Continuous Surface Embeddingscited by 79
  34. 2021Shelf-Supervised Mesh Prediction in the Wildcited by 65
  35. 2017MonoPerfCap: Human Performance Capture from Monocular Videocited by 63
  36. 2020MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Videocited by 62
  37. 2021DOVE: Learning Deformable 3D Objects by Watching Videoscited by 61
  38. 2022CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphismcited by 56

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 21; 2025: 12; 2026: 5.

46 papers.

Sort:

By area:

  1. 2023GART: Gaussian Articulated Template Modelscited by 121
  2. 2024GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creationcited by 99
  3. 2025ArtGS: Building Interactable Replicas of Complex Articulated Objects via Gaussian Splattingcited by 67
  4. 20244Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Modelscited by 64
  5. 2024Learning the 3D Fauna of the Webcited by 58
  6. 2024REACTO: Reconstructing Articulated Objects from a Single Videocited by 45
  7. 2024Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfelscited by 38
  8. 2023DynPoint: Dynamic Neural Point For View Synthesiscited by 37
  9. 2023AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillationcited by 28
  10. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  11. 2024Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videoscited by 26
  12. 2024DRiVE: Diffusion-based Rigging Empowers Generation of Versatile and Expressive Characterscited by 24
34 more
  1. 2023ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collectionscited by 22
  2. 2024InterTrack: Tracking Human Object Interaction without Object Templatescited by 22
  3. 2023Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videoscited by 21
  4. 2024Diffusion Priors for Dynamic View Synthesis from Monocular Videoscited by 18
  5. 2025VideoArtGS: Building Digital Twins of Articulated Objects from Monocular Videocited by 15
  6. 2025RigGS: Rigging of 3D Gaussians for Modeling Articulated Objects in Videoscited by 14
  7. 2024A Survey on 3D Human Avatar Modeling -- From Reconstruction to Generationcited by 13
  8. 2025Articulated Kinematics Distillation from Video Diffusion Modelscited by 13
  9. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  10. 2023DreaMo: Articulated 3D Reconstruction From A Single Casual Videocited by 12
  11. 2024AWOL: Analysis WithOut synthesis using Languagecited by 12
  12. 2024Sync4D: Video Guided Controllable Dynamics for Physics-Based 4D Generationcited by 12
  13. 2024DressRecon: Freeform 4D Human Reconstruction from Monocular Videocited by 12
  14. 2024S3O: A Dual-Phase Approach for Reconstructing Dynamic Shape and Skeleton of Articulated Objects from Single Monocular Videocited by 11
  15. 2024RHOBIN Challenge: Reconstruction of Human Object Interactioncited by 10
  16. 2024DualPM: Dual Posed-Canonical Point Maps for 3D Shape and Pose Reconstructioncited by 10
  17. 2024Learning Implicit Representation for Reconstructing Articulated Objectscited by 9
  18. 2025ART: Articulated Reconstruction Transformercited by 9
  19. 2025HoliGS: Holistic Gaussian Splatting for Embodied View Synthesiscited by 7
  20. 20254D-Animal: Freely Reconstructing Animatable 3D Animals from Videoscited by 6
  21. 2025DIMO: Diverse 3D Motion Generation for Arbitrary Objectscited by 6
  22. 2025Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Fieldcited by 4
  23. 2023Virtual Pets: Animatable Animal Generation in 3D Scenescited by 3
  24. 2026Articulat3D: Reconstructing Articulated Digital Twins From Monocular Videos with Geometric and Motion Constraintscited by 3
  25. 2023MorpheuS: Neural Dynamic 360deg Surface Reconstruction from Monocular RGB-D Videocited by 2
  26. 2024XHand: Real-time Expressive Hand Avatarcited by 2
  27. 2024TFS-NeRF: Template-Free NeRF for Semantic 3D Reconstruction of Dynamic Scenecited by 2
  28. 2025Advances and Trends in the 3D Reconstruction of the Shape and Motion of Animalscited by 2
  29. 20264DEquine: Disentangling Motion and Appearance for 4D Equine Reconstruction from Monocular Videocited by 2
  30. 2025MoReMouse: Monocular Reconstruction of Laboratory Mousecited by 1
  31. 2024Hierarchically Structured Neural Bones for Reconstructing Animatable Objects from Casual Videoscited by 0
  32. 2026CausalGS: Learning Physical Causality of 3D Dynamic Scenes with Gaussian Representationscited by 0
  33. 2026ArtiTwinSplat: Interactable Digital Twin Reconstruction via Gaussian Splatting from RGB-D videoscited by 0
  34. 2026Progressive Pose-Guided 4D Animal Reconstruction from Monocular Videocited by 0