mlsanityData to 14 Oct 2026

Animal Avatars: Reconstructing Animatable 3D Animals from Casual Videos

arXiv 2403.17103 ↗First posted 25 Mar 2024cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

We present a method to build animatable dog avatars from monocular videos. This is challenging as animals display a range of (unpredictable) non-rigid movements and have a variety of appearance details (e.g., fur, spots, tails). We develop an approach that links the video frames via a 4D solution that jointly solves for animal's pose variation, and its appearance (in a canonical pose). To this end, we significantly improve the quality of template-based shape fitting by endowing the SMAL parametric model with Continuous Surface Embeddings, which brings image-to-mesh reprojection constaints that are denser, and thus stronger, than the previously used sparse semantic keypoint correspondences. To model appearance, we propose an implicit duplex-mesh texture that is defined in the canonical pose, but can be deformed using SMAL pose coefficients and later rendered to enforce a photometric compatibility with the input video frames. On the challenging CoP3D and APTv2 datasets, we demonstrate superior results (both in terms of pose estimates and predicted appearance) to existing template-free (RAC) and template-based approaches (BARC, BITE).

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 42 of its 58 references that are in the map. First posted on arXiv 25 Mar 2024.

Builds on

42 of 58 references.

References · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  6. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,465
  7. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  8. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  9. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 494
  10. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  11. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  12. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
30 more
  1. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  2. 2021Animatable Neural Radiance Fields for Modeling Dynamic Human Bodiescited by 361
  3. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 350
  4. 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 312
  5. 2022NeuMan: Neural Human Radiance Field from a Single Videocited by 244
  6. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 221
  7. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 197
  8. 2020STAR: Sparse Trained Articulated Human Body Regressorcited by 194
  9. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 153
  10. 2023Vid2Avatar: 3D Avatar Reconstruction from Videos in the Wild via Self-supervised Scene Decompositioncited by 134
  11. 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 116
  12. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 112
  13. 2021Animatable Neural Radiance Fields from Monocular RGB Videoscited by 109
  14. 2018Creatures great and SMAL: Recovering the shape and motion of animals from videocited by 84
  15. 2020Continuous Surface Embeddingscited by 80
  16. 2017AI Challenger : A Large-scale Dataset for Going Deeper in Image Understandingcited by 73
  17. 2022ViTPose++: Vision Transformer for Generic Body Pose Estimationcited by 65
  18. 2014Category-Specific Object Reconstruction from a Single Imagecited by 56
  19. 2023AG3D: Learning to Generate 3D Avatars from 2D Image Collectionscited by 55
  20. 2022BARC: Learning to Regress 3D Dog Shape from Images by Exploiting Breed Informationcited by 51
  21. 2023Reconstructing Animatable Categories from Videoscited by 47
  22. 20203D Bird Reconstruction: a Dataset, Model, and Shape Recovery from a Single Viewcited by 40
  23. 2020RGBD-Dog: Predicting Canine Pose from RGBD Sensorscited by 35
  24. 2021Birds of a Feather: Capturing Avian Shape Models from Imagescited by 30
  25. 2022Common Pets in 3D: Dynamic New-View Synthesis of Real-Life Deformable Categoriescited by 29
  26. 2023Learning Neural Duplex Radiance Fields for Real-Time View Synthesiscited by 28
  27. 2018Who Let The Dogs Out? Modeling Dog Behavior From Visual Datacited by 20
  28. 2021Learning monocular 3D reconstruction of articulated categories from motioncited by 18
  29. 2021Coarse-to-fine Animal Pose and Shape Estimationcited by 16
  30. 2023APTv2: Benchmarking Animal Pose Estimation and Tracking with a Large-scale Dataset and Beyondcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 9; 2026: 14.

26 papers.

Sort:

By area:

  1. 2024AniMer: Animal Pose and Shape Estimation Using Family Aware Transformercited by 20
  2. 2024Generative Zoocited by 12
  3. 2025AniMer+: Unified Pose and Shape Estimation Across Mammalia and Aves via Family-Aware Transformercited by 7
  4. 20254D-Animal: Freely Reconstructing Animatable 3D Animals from Videoscited by 6
  5. 2025DIMO: Diverse 3D Motion Generation for Arbitrary Objectscited by 6
  6. 2025LIM: Large Interpolator Model for Dynamic Reconstructioncited by 5
  7. 2025ReVision: Refining Video Diffusion with Explicit 3D Motion Modelingcited by 5
  8. 2025Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Fieldcited by 5
  9. 2025DogMo: A Large-Scale Multi-View RGB-D Dataset for 4D Canine Motion Recoverycited by 4
  10. 2026NeuralFur: Animal Fur Reconstruction From Multi-View Imagescited by 4
  11. 2025Pose Splatter: A 3D Gaussian Splatting Model for Quantifying Animal Pose and Appearancecited by 2
  12. 20264DEquine: Disentangling Motion and Appearance for 4D Equine Reconstruction from Monocular Videocited by 2
14 more
  1. 2024RatBodyFormer: Rat Body Surface from Keypointscited by 1
  2. 2025MoReMouse: Monocular Reconstruction of Laboratory Mousecited by 1
  3. 2026MorphGS: Morphology-Adaptive Articulated 3D Motion Transfer from Videoscited by 1
  4. 2026SPRig: Self-Supervised Pose-Invariant Rigging from Mesh Sequencescited by 1
  5. 2026BigMaQ: A Big Macaque Motion and Animation Dataset Bridging Image and 3D Pose Representationscited by 1
  6. 2026SMAL-pets: SMAL Based Avatars of Pets from Single Imagecited by 1
  7. 2026PRIMA: Boosting Animal Mesh Recovery with Biological Priors and Test-Time Adaptationcited by 1
  8. 2026InterPet4D: A Multimodal 4D Human-Pet Interaction Dataset for Pet Motion Generationcited by 1
  9. 2026Kirin: Animal Motion Generation from In-the-Wild Videocited by 1
  10. 2026DogWeave: High-Fidelity 3D Canine Reconstruction from a Single Image via Normal Fusion and Conditional Inpaintingcited by 0
  11. 2026Progressive Pose-Guided 4D Animal Reconstruction from Monocular Videocited by 0
  12. 2026CORGI: Consistency-Aware 3D Dog Reconstruction from a Single Image in the Wildcited by 0
  13. 2026AnimalLift: Reconstructing Animatable 3D Animals from a Single Image by Learning Canonical Shape, Texture, and Fur Mapscited by 0
  14. 2026ORMA: Optimization-based Monocular 4D Reconstruction of Articulated Animalscited by 0